Medical AR

Intelligent Mixed Reality in Healthcare Applications

In close collaboration with clinicians, Duke I3T Lab has been demonstrating applications of next-generation intelligent MR in a wide range of healthcare scenarios.  

AR in surgical applications: In collaboration with two practicing neurosurgeons and several neurosurgical trainees advised by them, we investigated opportunities for intra-operative neurosurgical AR guidance, as well as employing AR for neurosurgical trainees' skills evaluation and training. 

AR-aided physical therapy (PT): With support from the National Science Foundation, we have been investigating the role AR exergames can play in supporting different kinds of physical therapy. We have formed a close collaboration with Duke University School of Nursing, as part of which we are currently developing an XR exergaming platform tailored to the needs of patients who are engaging in physical therapy while being treated in an Intensive Care Unit (ICU). 

Image
Sensing-integrated phantom for mixed reality-guided craniostomy
Sensing-integrated phantom for mixed reality-guided craniostomy 

Duke I3T Lab is associated with the Duke Center for Computational and Digital Health Innovation. PI Gorlatova serves as the Center's Associate Director of Extended Reality (XR). 

Recent and selected publications

[ToAppearCIN2025] A. Frith, A. Mall, D. Streisfeld, K. Swaringen, E. Escobar, B. B. Granger, and M. Gorlatova, Human Centered Design of a Virtual Reality Intervention to Promote Early Mobility in a Cardiothoracic Intensive Care Unit. To appear in CIN: Computers, Informatics, Nursing, 2025. 

[NeurosurgicalFocus24] S. Eom, T. Ma, N. Vutakuri, T. Hu, A. P. Haskell-Mendoza, D. W. Sykes, M. Gorlatova, and J. Jackson. Mixed Reality-Guided Twist-Drill Craniostomy Improves the Accuracy of External Ventricular Drain Placement. Neurosurgical Focus, special issue on Mixed Reality in Neurosurgery. Vol. 56, No. 1, Jan. 2024. 

[XRHealth23] S. Eom, S. Kim, Y. Jiang, R. Chen, A. Roghanizad, M. Z. Rosenthal, J. Dunn, M. Gorlatova, Investigation of Thermal Perception and Emotional Response in Augmented Reality using Digital Biomarkers: A Pilot Study, in Proc. IEEE XR for Healthcare and Wellbeing Workshop (co-located with IEEE VR), 2023. 

[ISMAR22a] S. Eom, D. Sykes, S. Rahimpour, and M. Gorlatova, NeuroLens: Augmented Reality-based Contextual Guidance through Surgical Tool Tracking in Neurosurgery, in Proc. IEEE ISMAR, Singapore, Oct. 2022 (21% acceptance rate). Demonstrated to members of US legislature as part of the 2023 National AI Research Institutes Congressional Showcase. 

Great thanks to our sponsors! 

This research is supported by the National Science Foundation.