PERFEX-I: confidence scores for image classification using decision trees

Thijs Eker; Ajaya Adhikari; Sabina B. van Rooij

doi:10.1117/12.3049363

13 November 2024 PERFEX-I: confidence scores for image classification using decision trees

Thijs Eker, Ajaya Adhikari, Sabina B. van Rooij

Proceedings Volume 13206, Artificial Intelligence for Security and Defence Applications II; 132060J (2024) https://doi.org/10.1117/12.3049363
Event: Security + Defence, 2024, Edinburgh, United Kingdom

Abstract

To be able to use machine learning models in practice, it is important to know when their predictions can be trusted. Confidence estimations can help end users to calibrate their trust, avoiding under- or over-reliance, and to decide when human interference is needed. In our work, we further develop the eXplainable AI (XAI) method PERformance EXplainer (PERFEX), which was originally proposed for tabular datasets. We adapt PERFEX such that it can be used to accurately estimate the image classifier confidence. This was done by applying the method on feature-reduced activation values of the last layer of image classification models. We coin this approach PERFEX-I. We show that PERFEX-I performs on par with existing methods for confidence estimation such as Temperature Scaling and Deep Ensembles. The Expected Calibration Error (ECE) on the ImageNet dataset is reduced from 6.83 to 1.71 for ResNet50 and from 8.84 to 1.44 for Swin-B compared to using the Softmax scores. Additionally, PERFEX-I groups images that may share common reasons for errors, and visual analysis of these groups can reveal patterns of the model’s behavior.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Thijs Eker, Ajaya Adhikari, and Sabina B. van Rooij "PERFEX-I: confidence scores for image classification using decision trees", Proc. SPIE 13206, Artificial Intelligence for Security and Defence Applications II, 132060J (13 November 2024); https://doi.org/10.1117/12.3049363

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available