Multi-modality fusion coupled with deep learning for improved outcome prediction in head and neck cancer

Arash Javanmardi; Mahdi Hosseinzadeh; Ghasem Hajianfar; Amir Hossein Nabizadeh; Seyed Masoud Rezaeijo; Arman Rahmim; Mohammadreza Salmanpour

doi:10.1117/12.2612833

4 April 2022 Multi-modality fusion coupled with deep learning for improved outcome prediction in head and neck cancer

Arash Javanmardi, Mahdi Hosseinzadeh, Ghasem Hajianfar, Amir Hossein Nabizadeh, Seyed Masoud Rezaeijo, Arman Rahmim, Mohammadreza Salmanpour

Author Affiliations +

Proceedings Volume 12032, Medical Imaging 2022: Image Processing; 120322I (2022) https://doi.org/10.1117/12.2612833
Event: SPIE Medical Imaging, 2022, San Diego, California, United States

Conference Poster

Abstract

Multi-level multi-modality fusion radiomics is a promising technique with potential to improve the prognostication of cancer. We aim to use advanced fusion techniques on PET and CT images coupled with deep learning (DL) to improve outcome prediction in head and neck squamous cell carcinoma (HNSCC). In our study, 408 HNSCC patients were included from The Cancer Imaging Archive (TCIA) in a multi-center setting. Prognostic outcomes (binary classification) included overall survival (OS), distant metastasis (DM), locoregional recurrence (LR), and progression free survival (PFS). We utilized a DL algorithm with a 17-layer 3D convolutional neural network (CNN) architecture. Prior to training, each image underwent min-max-normalization, image-augmentation by using random rotations (0-20°) to improve the performance and generalizability of our model and followed by 5-fold-cross-validation. We employed 12 datasets, including CT, PET, and 10 image-level fused datasets. The best OS performance was achieved via Discrete-wavelet-transform (DWT) resulting in mean accuracy of 0.93±0.06. The best DM score was achieved via ratio of low-pass pyramid (RLPP), resulting in an accuracy of 0.95±0.02. Optimal LR and PFS scores were achieved using DWT and RLPP for LR, and Laplacian pyramid for PFS, resulting in accuracies of 0.90-0.92. Comparatively, when using a machine learning framework instead of deep learning, we obtained scores of 0.83, 0.90, and 0.87 for the prediction of OS, DM, and LR. Our study demonstrates that our multi-modality fusion techniques performed better than using standalone PET or CT in prognostication of HNSCC patients and that a high level of accuracy can be achieved for prognostication of HNSCC patients when combining multi modality fusion techniques with DL.

Citation Download Citation

Arash Javanmardi, Mahdi Hosseinzadeh, Ghasem Hajianfar, Amir Hossein Nabizadeh, Seyed Masoud Rezaeijo, Arman Rahmim, and Mohammadreza Salmanpour "Multi-modality fusion coupled with deep learning for improved outcome prediction in head and neck cancer", Proc. SPIE 12032, Medical Imaging 2022: Image Processing, 120322I (4 April 2022); https://doi.org/10.1117/12.2612833

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available