We present results comparing black-box and physics-guided neural network architectures for hyperspectral target identification. Specifically, our physics-guided neural networks operate on at-sensor overhead long-wave infrared hyperspectral imaging radiances to predict not only the material class, but also physically-meaningful quantities of interest, such as the atmospheric transmission factor, the temperature, and the underlying material emissivity. In this way, our models are decoupled from traditional preprocessing routines and provide independently verifiable and interpretable quantities alongside the class predictions. We compare our physics-guided models to more traditional black-box models with respect to classification accuracy and representational similarity, and assess performance in predicting physical quantities across a variety of training schemes.
Longwave Infrared hyperspectral images (LWIR HSI) are a powerful data source for various applications in national security and environmental monitoring. A promising area for applying machine learning to LWIR HSI data is for gas plume identification from remote sensing platforms. However, a significant practical difficulty in using HSI for this task is the ability to estimate and remove the background spectra underlying a detected gas plume. Typically, one estimates a covariance matrix and a mean spectrum using all pixels from an image to whiten the pixels of interest before substance identification. We propose using image segmentation to define local regions to perform this whitening. We investigate both local and global estimation of the covariance and mean spectrum, and find that using the global covariance and local mean increases prediction confidence using our deep learning classification model. Using an airborne LWIR capture of the Los Angeles basin, we investigate performance increases by generating an ensemble of random marker-based Watershed segmentations. The ensemble of segmentations provides nuanced mean estimates for each pixel in the gas plume, leading to increased machine learning classification confidence. This method shows significant promise for improving machine learning classification applied to real-world HSI collects.
In remote sensing image analysis, change detection approaches typically compare two images captured by the same airborne or spaceborne sensor at different points in time. However, as airborne and spaceborne imaging platforms have become increasingly more accessible, the variety of sensor designs has grown in tandem. The ability to combine these multi-modal remote sensing images for change detection would provide a far more frequent view of the earth, but traditional approaches are challenged by the intrinsic data variation across sensor designs. The recently introduced multi-sensor anomalous change detection (MSACD) framework addresses this challenge by using a data-driven machine learning approach that can effectively account for differences in sensor modality and design, and does not require any signal resampling of the pixels. This flexible framework enables the use of satellite image pairs from different sensor platforms. Here, we perform experiments to further evaluate the efficacy of the MSACD change detection framework; these experiments include augmenting the images with engineered features that seek to increase the mutual information of the image backgrounds and, in turn, better emphasize the anomalous changes. While these initial results are demonstrated on same-sensor spectral data, the experiments naturally extend to the multi-sensor domain.
Machine learning approaches, such as deep neural networks, have shown recent success for target detection and identification problems in hyperspectral imagery. However, when deployed “in the wild,” there are no guarantees about the behavior of these black box algorithms when encountering new materials or environmental conditions that were not part of the training data. In addition, neural networks typically lack properties of linear identification methods in that their predictions tend to select a single class with high confidence even when there are multiple classes that could match a given input spectrum. To provide estimates of confidence in neural network predictions (i.e., target identifications) and to produce indicators of uncertainty, we apply stateof-the-art uncertainty quantification techniques to neural networks trained on hyperspectral data. Specifically, we assess recently proposed methods from the machine learning community including Monte Carlo dropout, ensembles of neural networks, and variational Bayesian neural networks. We report not only the accuracy of the resulting model-averaged networks on in-distribution data, but also the usefulness of uncertainty metrics on noisy or out-of-distribution data. We also compare ensemble neural network target identification results to a linear method on airborne long-wave infrared (LWIR) hyperspectral data with real targets. Finally, we offer some guidelines for applying these methods to hyperspectral target detection/identification problems.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.