How do you explain something you can’t see?

“The inner ear is one of the toughest topics to teach because it is anatomically very complex, very small and buried in the dense petrous bone,” says Professor Emerita Pat Stewart.

So Andrea Zariwny, a biomedical communications graduate student, tackled this teaching problem with an augmented reality iPad app and 3D-printing to visualize the anatomy of the inner ear.

Here’s how it works: Launch Zariwny’s app. Hold the iPad over Zariwny’s illustration of the petrous bone. The illustration acts as a glyph—a high-contrast image recognized by the tablet’s camera—and triggers a 3-dimensional digital model on top of the illustration. Look through the digital model to the illustration below and see the cochlea of the inner ear or the spiral ganglion lift off the page.

“The illustration on its own is an informative, didactic tool,” says Professor Marc Dryer, Zariwny’s research supervisor. “But augmented with another layer of information, it becomes a strong educational feature that enhances the learning experience and engages student attention.”

Zariwny and her content advisor, Stewart, used micro CT scanning to create medical images from real anatomical evidence. Zariwny took the data into a medical image viewer and extracted 3D digital models of the petrous bone, and the cochlea and spiral ganglion—internal structures of the inner ear. She retopologized or ‘cleaned up’ the models and took them into gaming software to create the 3-dimensional digital models.

After Zariwny, who has a background in industrial design, created the augmented textbook illustration, she wanted to work with 3D printing. “I thought augmented reality could be an even more useful tool combined with a physical model,” says Zariwny.

Zariwny 3D-printed two models of the petrous bone—one whole and one sliced in half. She created modules for her app that “ghost” the internal structure of the inner ear over the physical models. The apps recognize, track and match the digital models to the position of the physical models. Move the models, or the iPad, and the digital models allow viewers access to different orientations and perspectives.

Zariwny’s tool accurately conveys the size and position of what is essentially space buried in bone, says Stewart. “The first time I saw the models, I thought, that’s it! Exactly what we need.”

Published in U of T Mississauga Magazine Fall 2013