Fusion of deep-learning models for multi-view image classification

Brian Maguire; Eleanor Seminerio

doi:10.1117/12.2665000

14 June 2023 Fusion of deep-learning models for multi-view image classification

Brian Maguire, Eleanor Seminerio

Proceedings Volume 12547, Signal Processing, Sensor/Information Fusion, and Target Recognition XXXII; 125470E (2023) https://doi.org/10.1117/12.2665000
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Abstract

Advances in Convolutional Neural Networks (CNN) have demonstrated state of the art performance in the tasks of image classification and object detection over the past decade. While significant progress has been made in development of more efficient networks, the computational and memory requirements still exceed practical limits in many applications. Additionally, the pose variability in such applications requires even larger training datasets for the network to generalize to all possible scenarios. The goal of this work is to develop an architecture for fusion of multiple views of a single target to provide robust classification with a lightweight backbone network used across all agents. Motivated by approaches to ensemble learning, we demonstrate that multiple weak learners with computationally efficient networks can combine to enhance classification accuracy. Three methods of fusion are considered: decision fusion, feature fusion, and multi-scale feature fusion. A novel network architecture is developed and implemented for each approach then trained and evaluated using synthetic data. For the feature fusion models, a custom training scheme is developed to minimize classification error while maintaining a common feature extraction backbone across agents. This conforms to a distributed classification use case where each agent has no prior knowledge of its position relative to target. Finally, we discuss the requirements for shared data of each approach in the context of applications with limited communication bandwidth.

Conference Presentation

Citation Download Citation

Brian Maguire and Eleanor Seminerio "Fusion of deep-learning models for multi-view image classification", Proc. SPIE 12547, Signal Processing, Sensor/Information Fusion, and Target Recognition XXXII, 125470E (14 June 2023); https://doi.org/10.1117/12.2665000

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available