Augmented reality visualization of hyperspectral imaging classifications for image-guided brain tumor phantom resection

James Huang; Martin Halicek; Maysam Shahedi; Baowei Fei

doi:10.1117/12.2549041

16 March 2020 Augmented reality visualization of hyperspectral imaging classifications for image-guided brain tumor phantom resection

James Huang, Martin Halicek, Maysam Shahedi, Baowei Fei

Author Affiliations +

Proceedings Volume 11315, Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling; 113150U (2020) https://doi.org/10.1117/12.2549041
Event: SPIE Medical Imaging, 2020, Houston, Texas, United States

Abstract

Wearable augmented reality (AR) is an emerging technology with enormous potential for use in the medical field, from training and procedure simulations to image-guided surgery. Medical AR seeks to enable surgeons to see tissue segmentations in real time. With the objective of achieving real-time guidance, the emphasis on speed produces the need for a fast method for imaging and classification. Hyperspectral imaging (HSI) is a non-contact, optical imaging modality that rapidly acquires hundreds of images of tissue at different wavelengths, which can be used to generate spectral data of the tissue. Combining HSI information and machine-learning algorithms allows for effective tissue classification. In this paper, we constructed a brain tissue phantom with porcine blood, yellow-dyed gelatin, and colorless gelatin to represent blood vessels, tumor, and normal brain tissue, respectively. Using a segmentation algorithm, hundreds of hyperspectral images were compiled to classify each of the pixels. Three segmentation labels were generated from the data, each with a different type of tissue. Our system virtually superimposes the HSI channels and segmentation labels of a brain tumor phantom onto the real scene using the HoloLens AR headset. The user can manipulate and interact with the segmentation labels and HSI channels by repositioning, rotating, changing visibility, and switching between them. All actions can be performed through either hand or voice controls. This creates a convenient and multifaceted visualization of brain tissue in real time with minimal user restrictions. We demonstrate the feasibility of a fast and practical HIS-AR technique for potential use of image-guided brain surgery.

Citation Download Citation

James Huang, Martin Halicek, Maysam Shahedi, and Baowei Fei "Augmented reality visualization of hyperspectral imaging classifications for image-guided brain tumor phantom resection", Proc. SPIE 11315, Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, 113150U (16 March 2020); https://doi.org/10.1117/12.2549041

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available