You know VR, AR, but what is spatial computing? – MedCity News

Brainlab Mixed Reality Viewer based on the Magic Leap headset

Augmented reality and virtual reality are terms that the healthcare industry is experimenting with using a variety of applications.

Now comes the next stage spatial computing.

One company at the forefront of this new technology is Magic Leap, but it has torecover from its reported woes. In an interview earlier this year,Jennifer Esposito, vice president of health for the Florida company, described thetechnology in detail.

When we talk about spatial computing, it is really the idea that the digital world and the physical world are fully interacting. They are aware of each other. There are things like persistence so if you believe something in the digital world, it will be there when you come back and there is this interactivity between the physical and the digital world. Thats how we try to define and differentiate it. We see it as a new computing platform using things like computer vision to make this merger of the physical and the digital possible.

It was a theme that Esposito returned to during a panel discussion at theHLTHconference in Las Vegas in late October.She described a wearable device that could track head position, voice, eye movement and gestures.

This feeds into a magicverse, in which any physical location can have a digital twin, creating new opportunities to enhance and overlay information.

It could be something as small as a room, she said. It could be a city. It could be a country. It could be a hospital. It could be an operating room.

And it is the operating room where Florida-based Magic Leap is hoping its collaboration with Brainlab will offer new possibilities in healthcare for better patient care, medical education and surgical planning.

Brainlab is a privately held, German medical technology company that makes software and hardware for treatments in radiosurgery and radiotherapy as well as other surgical fields including neurosurgery, ENT, CMF, spine and trauma. Per the companys website, Brainlabs has nearly 13,000 systems installed in more than 100 countries.

In November, the company announced that it waslaunching what it calledMixed Reality Viewer, which is awaiting FDA clearancethat can bring spatial computing into daily clinical practice for surgical plan review, medical student training and patient consultation.

The viewer from Brainlab uses the head-mounted display from Magic Leap called MagicMagic Leap One and people can see scans and anatomical structures and manipulate thereby freeing people from seeing 2D scans on screens.

Esposito described the potential of this new tool in an interview before the announcement.

One of the first and most obvious use cases for spatial computing is simply enhanced visualization so bringing in complex data together and making it easier for people to understand and so this is a great example of that because youll be able to see radiology images and other information and make it as big or as small as you want. You can consult with your colleagues about it and I will see the same thing as you when seeing it from a different angle. Think about two or three people all wearing the same Magic Leap headset and they are participating in this 3D Dicom viewer capability. Multiple surgeons or other staff and even family members being able to see those images in 3D. Whats important to note here is that we are working on the Brainlab platform and they have an absolutely amazing surgical planning and navigation system. So its taking all of their underlying software capabilities.

But while this sounds very fascinating, there are huge cost pressures in the healthcare world. Will hospitals feel the need to invest in such a device?

I think that while some of these things will require capital investment, the hope and the potential is that these are technologies that help address the Triple Aim and really augment the clinical workforce and in the long run improve value and reduce costs, Esposito said.

Brainlabs founder and CEO, Stefan Vilseymer, who has known the founder of Magic Leap Ron Abovitz, who founded and sold Mako Surgical to Stryker for two decades was impressed when he reviewed the tech in Magic Leap.

I have to say that when I first saw what he was trying to do, I was really blown away by the possibilities and they were way beyond what I would have anticipated to be possible, Vilseymer said in a recent interview.

He explained why the partnership with Magic Leap can transform even what Brainlab currently does in operating rooms.

Today in order to provide all our digital tools, we have our relatively complicated computer systems that are basically huge terminals that you wheel into the operating room. So they are computers, couple of screens, a set of cameras and they are typically products where the hardware itself costs $100K $200K. Thats pretty much the same with all of our competitors.

What Magic Leap really allows you to do we can virtualize this huge hardware effort because you can take software out of the box and place it into the physical space of the operating room. So the information is on the screen but it is anywhere everywhere you need it.

But even outside the arena of surgery, this technology could liberate people from brick and mortar clinics, providing a more detailed, and hopefully more satisfying, version of telehealth.

So, you have someone in an automated car, Esposito said at HLTH. They have the ability to potentially experience some sort of primary care or urgent care visitMaybe theyre on their way to work. Maybe theyre on their way to the hospital.

In another scenario, Esposito described bike riders receiving more robust information about their surroundings and perhaps even their physical performance. They might even get health tips.

That could be the patient receiving information about how what theyre doing is impacting their health and maybe suggestions on things they could do to change, she said.

The futuristic vision of Magic Leap is quite intriguing, but there are other companies in the field, including Microsoft with its HoloLens mixed reality headset. Whether Magic Leap can prevail and have an impact in both the gaming and healthcare worlds will depend on customer demand for its headset. While the company has raised an eye-popping $2.6 billion in seven funding rounds, according to Crunchbase, recent news reports seem to suggest that the company is facing a cash crunch and key executives have left. The company has also reportedly put up its patent portfolio as security for the latest funding round to close and is dealing with less-than-expected sales of its Magic Leap One headset.

Josh Baxt contributed to this story.

Photo: Brainlab

The rest is here:

You know VR, AR, but what is spatial computing? - MedCity News