Kernel-weighted contribution: a method of visual attribution for 3D deep learning segmentation in medical imaging

Sean Mullan; Milan Sonka

doi:10.1117/1.JMI.10.5.054001

7 September 2023 Kernel-weighted contribution: a method of visual attribution for 3D deep learning segmentation in medical imaging

Sean Mullan, Milan Sonka

Author Affiliations +

Journal of Medical Imaging, Vol. 10, Issue 5, 054001 (September 2023). https://doi.org/10.1117/1.JMI.10.5.054001

Abstract

Purpose

Explaining deep learning model decisions, especially those for medical image segmentation, is a critical step toward the understanding and validation that will enable these powerful tools to see more widespread adoption in healthcare. We introduce kernel-weighted contribution, a visual explanation method for three-dimensional medical image segmentation models that produces accurate and interpretable explanations. Unlike previous attribution methods, kernel-weighted contribution is explicitly designed for medical image segmentation models and assesses feature importance using the relative contribution of each considered activation map to the predicted segmentation.

Approach

We evaluate our method on a synthetic dataset that provides complete knowledge over input features and a comprehensive explanation quality metric using this ground truth. Our method and three other prevalent attribution methods were applied to five different model layer combinations to explain segmentation predictions for 100 test samples and compared using this metric.

Results

Kernel-weighted contribution produced superior explanations of obtained image segmentations when applied to both encoder and decoder sections of a trained model as compared to other layer combinations (p < 0.0005). In between-method comparisons, kernel-weighted contribution produced superior explanations compared with other methods using the same model layers in four of five experiments (p < 0.0005) and showed equivalently superior performance to GradCAM++ when only using non-transpose convolution layers of the model decoder (p = 0.008).

Conclusion

The reported method produced explanations of superior quality uniquely suited to fully utilize the specific architectural considerations present in image and especially medical image segmentation models. Both the synthetic dataset and implementation of our method are available to the research community.

Citation Download Citation

Sean Mullan and Milan Sonka "Kernel-weighted contribution: a method of visual attribution for 3D deep learning segmentation in medical imaging," Journal of Medical Imaging 10(5), 054001 (7 September 2023). https://doi.org/10.1117/1.JMI.10.5.054001

Received: 1 March 2023; Accepted: 23 August 2023; Published: 7 September 2023

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $24.00

Non-members: $28.00 ADD TO CART

JOURNAL ARTICLE
17 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image segmentation

Data modeling

Convolution

Medical imaging

Visualization

3D modeling

Education and training

Purpose

Approach

Results

Conclusion

Show All Keywords

Keywords/Phrases

Search In:

Publication Years