You’ve heard of virtual reality, but what about augmented reality? The name may seem unfamiliar, but there’s a good chance you’ve used it without realizing it. You might have even played a game built on it. A popular example is Pokémon Go, a smartphone app that lets you roam your neighborhood or community to “catch” wild Pokémon that appear on screen as if they’ve appeared right in front of you.
Virtual reality and augmented reality are closely linked. Either way, you can participate in a computerized virtual world that appears in front of you, whether on your phone screen or on a headset.
But there is a really important difference. Virtual reality is immersive. This means that it allows you to immerse yourself in a totally imaginary world, with sight and sound. While you can engage in it and it may even sound realistic or realistic, virtual reality exists entirely outside of all that is real.
What makes augmented reality special and more useful in medicine is that it merges virtual images with the real world, including real objects and real people. This means that your surgeon, doctor, or nurse could see things in front of them using augmented reality that they otherwise couldn’t – like veins running through your arm, a broken bone, a brain tumor, an x-ray, or health records – – without taking their attention away from you to look at another screen.
“[In augmented reality], there is something floating about you in the room you are in, ”says Greg Dorsainville, head of immersive computing at NYU Langone Health. “You are in your world and the data is superimposed on the elements of the world.”
Augmented reality in the clinic
As technology improves, augmented reality has a lot of potential to impact medicine. Augmented reality can find its way into the primary care clinic, operating room, emergency room, and dental office. Doctors could use it, for example, to plan plastic surgeries and other complex operations. They could also use it to guide them during surgeries of all kinds.
Augmented reality involves the use of software technology to improve the real-world environment, says Thomas Hopkins, MD, medical director of a company called AccuVein. In the field of health, there are many possibilities to improve the technique to educate people, improve procedures and take care of people.
AccuVein is one example. The handheld device uses laser technology to “see” through your skin and into your veins. It is intended to make it easier for doctors, nurses or others to find a vein to draw blood or place an IV. Hopkins, who is also an assistant professor of anesthesiology at Duke University Health System, says there is already data showing that the augmented reality device is making things easier for both doctors and the people they treat. Healthcare providers are more likely to get it right the first time, reducing the risk of additional needle sticks while saving time.
Augmented reality in the operating room
Augmented reality is just starting to find its way into higher stakes healthcare. In June 2020, neurosurgeons at Johns Hopkins University in Baltimore announced the first augmented reality surgery. A doctor used it to place six screws during spinal surgery to relieve severe back pain. Soon after, they used it to remove a tumor from someone else’s spine.
Augmented reality technology included a headset with a screen that doctors could see through the person. This allowed them to project X-ray or CT scan images, for example, on the body, to see both at the same time. As long as these images are perfectly aligned, it is as if surgeons have x-ray vision.
Timothy Witham, MD, the Johns Hopkins neurosurgeon who led the surgeries, says the technology acts like a GPS, guiding the path into the operating room. The alternative is for doctors to do their best to place the screws “freehand”.
“There are advantages and disadvantages to all techniques,” he says. “The main pro [of doing it freehand] it is the fastest. But it requires the best knowledge of anatomy and experience to do it almost blindly based on anatomical landmarks.
It is possible to use an x-ray, but it means radiation exposure for you and your surgeons. Witham says he now regularly uses augmented reality tools. He has found that this achieves 98% accuracy, which is at least as good as any other method.
“I feel more comfortable using it,” he says. The anxiety level is down a notch in terms of where to put tools. There is more confidence in doing it, and it is useful. Ultimately, this benefits patients.
One of the reasons for its early use in spinal surgery is that the rigid structure of the spine facilitates the correct binding of computerized images to the body. Witham says it would be more difficult to use for operations on other areas like the stomach or chest, where movement makes it much more difficult to align the virtual world with the real world.
Augmented reality in medical education
While many applications in medicine are not yet ready, augmented reality is already regularly used by doctors and nurses in training.
“If he does not follow correctly [in the operating room], you would have mistakes, ”said Dorsainville. “In teaching and learning, you don’t need that level of precision.”
At NYU Langone, students and faculty use augmented reality tools to access learning materials and learn about human anatomy. For example, they can run a detailed 3D model of the heart on a computer or phone. In the human anatomy lab, they can scan a QR code – those wavy squares you scan in a restaurant for a menu – on a corpse they’re studying to get more information about what they’re seeing, including videos of relevant surgeries or conferences.
“It’s really very early,” says Dorsainville. “But when augmented reality is done right, it’s really special.”
Healthcare tends to lag behind when it comes to technology adoption, Hopkins says.
There are good reasons for this. Doctors need time to prove that new technology is safe and benefits them and the people they treat. There are also issues with costs and integrating new technologies into existing workflows.
Despite the challenges, experts say they expect to see more applications of augmented reality in medicine. But it remains to be seen which technologies and uses will prove their worth in the clinic. You are likely to see rapid changes as tech companies strive to develop new uses for augmented reality, inside and outside the clinic.
“It’s a big front to see who will write the future on this type of technology,” says Dorsainville.
Our sincere thanks to