Remotely monitoring coastal ecosystems is critically enabled by hyperspectral imaging (HSI). HSI is corrupted by the water column in the scene, imparting a fingerprint on the imagery that is characteristic of the scene. Removing these effects from the scene requires (i) radiative transfer modeling and simulation, or approximations thereof, and (ii) knowledge of the sub-surface topology, i.e. bathymetry. This work explores using a novel physics-informed machine learning (ML) framework for joint estimation of bathymetry and corrected spectra. We formulate the problem of transmission of spectra through a medium as a tunable Ordinary Differential Equation (ODE). Here, the learned ODE is a surrogate model for transmission of radiation while the integration path length provides a measure of column depth; i.e. bathymetry. In this work, we show that inverting this ODE-based relationship provides reasonable estimates for bathymetry. We demonstrate a proof-of-concept of this inversion using a HSI scene of Enrique Reef on the southwest coast of Puerto Rico.
Given the scale and complexity of forthcoming HSI data, producing labeled datasets at the scale required to improve state-of-the-art performance is impractical and prohibitively costly. Unsupervised pre-training algorithms have revolutionized deep learning for natural language processing and computer vision by tapping into vast troves of unlabeled data, but these advances have seen little adoption in the HSI domain. We present some early results from self-supervised pre-training for hyperspectral imagery using masked auto-encoders early and compare different pre-training approaches and masking techniques; specifically masking size, dimension (spatial, spectral, both), mask fraction, and mask coherence (spatially independent or consistent). We summarize our lessons learned and highlight the most promising approaches towards building a foundation model for hyperspectral data.
Atmospheric correction is the process for removing atmospheric effects from spectral data; a necessary step for recovering salient spectral properties. The complex interactions between the atmosphere and light are dominated by absorbance and scattering physics. Existing methods for modeling atmospheric interactions typically rely on deep knowledge of relevant environmental conditions and high-fidelity numerical simulations of the governing physics in order to obtain accurate estimates of these effects. Additionally, existing approaches often require a subject matter expert for pre/post-processing of the data. Model-based approaches for removing atmospheric effects struggle in situations where such domain expertise is not available, and require significant human effort and computational power even when that expertise is available. In contrast, we propose a data-driven approach the uses Neural Differential Equations (NDEs) to accurately learn the interactions between electromagnetic radiation and the atmospheric without access to location specific environmental information. Once trained, the NDE can be applied bi-directionally; to apply or remove atmospheric effects. We demonstrate the effectiveness and utility of these techniques on an example multi-spectral scene.
Predictive maintenance refers to the ability to predict when machinery or systems need to be maintained. Making an accurate prediction is quite challenging given the costs for both over-estimating (unnecessary maintenance and reduction in availability of assets) and under-estimating (untimely breakdowns and possible loss of equipment or lives). To address these challenges researchers were able to develop new approaches for analyzing oil samples taken extracting samples from oil-wetted machinery that may provide information critical to developing predictive capabilities. We consider the problem from both supervised (though data limited) and unsupervised approaches and provide a first look into a data driven approach for identification of condition indicators. Through this work we identify a collection of candidate features that can form the basis of condition indicators for both a high level discrimination of failure vs. normal operation as well as a set for potential failure mode identification. Finally, we present an anomaly detection framework for detecting failures which can be a viable solution for an onboard analysis tool in deployed systems.
We investigate an anomaly detection framework that uses manifold-based distances within the existing skeleton kernel principle component analysis (SkPCA) manifold-learning technique. SkPCA constructs a manifold from the an adjacency matrix built using a sparse subsample of the data and a similarity measure. In anomaly detection the relative abundance of the anomalous class is rare by definition and in practice anomalous samples are unlikely to be randomly selected for inclusion in the sparse data subsample. Thus, anomalies should not be well modeled by the SkPCA-constructed model. Here, we consider alternative distance measures based on viewing spectral pixels as points in projective space, that is, each pixel is a 1-dimensional line through the origin. Chordal and geodesic distances are computed between hyperspectral pixels and detection performance leveraging these distances is compared to alternative anomaly detection algorithms. In addition, we introduce Ensemble SkPCA which utilizes the ensemble of mean, normalized detection scores corresponding to multiple randomly generated skeletons. For acceptable false alarm tolerances, the ensemble detection score derived from chordaland geodesic-based methods achieves higher probability of detection than Euclidean distance-based Ensemble SkPCA or the benchmark RX algorithm.
KEYWORDS: Data modeling, Image fusion, Hyperspectral imaging, Principal component analysis, Multispectral imaging, Data fusion, Detection and tracking algorithms
Kernel-based methods for anomaly detection have recently shown promising results - surpassing those of model-based statistical methods. This success is due in part to the distribution of the non-anomalous data failing to conform to the distribution model assumed by model-based statistical methods. Alternatively, the skeleton kernel principle component analysis anomaly detector (sKPCA-AD) assumes that a better background model can be learned by constructing a graph from a small, randomly sampled subset of the data (a skeleton). By definition, anomalies are rare and thus the sampling is assumed to be comprised chiefly of non-anomalous samples and correspondingly the learned graph models the background. Error magnitudes in the models' representation of data from the full data set are used as an anomaly measure. Additionally, the smaller skeleton sample makes kernel methods computationally feasible for hyperspectral images.
The sKPCA-AD has proven successful using unordered spectral pixel data, however, anomalies are often larger objects composed of many neighboring pixels. In this paper we show that fusing spatial information derived from a panchromatic image with spectral information from a hyper/multispectral image can increase the accuracy of the sKPCA-AD. We accomplish this by creating several joint spectral-spatial kernels that are then used by the sKPCA-AD to learn the underlying background model. We take into account the variability introduced by the random subsampling by showing averaged results and variance over several skeletons. We test our methods on two representative datasets and our results show improved performance with one of the proposed joint kernel methods.
We investigate an anomaly detection framework that leverages manifold learning techniques to learn a background model. A manifold is learned from a small, uniformly sampled subset under the assumption that any anomalous samples will have little effect on the learned model. The remaining data are then projected into the manifold space and their projection errors used as detection statistics. We study detection performance as a function of the interplay between sub-sampling percentage and the abundance of anomalous spectra relative to background class abundances using synthetic data derived from field collects. Results are compared against both graph-based and traditional statistical models.
A 16-band plenoptic camera allows for the rapid exchange of filter sets via a 4x4 filter array on the lens's front aperture. This ability to change out filters allows for an operator to quickly adapt to different locales or threat intelligence. Typically, such a system incorporates a default set of 16 equally spaced at-topped filters. Knowing the operating theater or the likely targets of interest it becomes advantageous to tune the filters. We propose using a modified beta distribution to parameterize the different possible filters and differential evolution (DE) to search over the space of possible filter designs. The modified beta distribution allows us to jointly optimize the width, taper and wavelength center of each single- or multi-pass filter in the set over a number of evolutionary steps. Further, by constraining the function parameters we can develop solutions which are not just theoretical but manufacturable. We examine two independent tasks: general spectral sensing and target detection. In the general spectral sensing task we utilize the theory of compressive sensing (CS) and find filters that generate codings which minimize the CS reconstruction error based on a fixed spectral dictionary of endmembers. For the target detection task and a set of known targets, we train the filters to optimize the separation of the background and target signature. We compare our results to the default 16 at-topped non-overlapping filter set which comes with the plenoptic camera and full hyperspectral resolution data which was previously acquired.
We investigate the parameters that govern an unsupervised anomaly detection framework that uses nonlinear techniques to learn a better model of the non-anomalous data. A manifold or kernel-based model is learned from a small, uniformly sampled subset in order to reduce computational burden and under the assumption that anomalous data will have little effect on the learned model because their rarity reduces the likelihood of their inclusion in the subset. The remaining data are then projected into the learned space and their projection errors used as detection statistics. Here, kernel principal component analysis is considered for learning the background model. We consider spectral data from an 8-band multispectral sensor as well as panchromatic infrared images treated by building a data set composed of overlapping image patches. We consider detection performance as a function of patch neighborhood size as well as embedding parameters such as kernel bandwidth and dimension. ROC curves are generated over a range of parameters and compared to RX performance.
We exploit manifold learning algorithms to perform image classification and anomaly detection in complex scenes involving hyperspectral land cover and broadband IR maritime data. The results of standard manifold learning techniques are improved by including spatial information. This is accomplished by creating super-pixels which are robust to affine transformations inherent in natural scenes. We utilize techniques from harmonic analysis and image processing, namely, rotation, skew, flip, and shift operators to develop a more representational graph structure which defines the data-dependent manifold.
We introduce a novel method for image fusion based on wavelet packets. Our ideas yield an approach for pan-sharpening low spatial resolution multispectral images with high spatial resolution panchromatic images. Two distinct algorithms for fusing are investigated, based on which wavelet packet coefficients are mixed. We evaluate our algorithm on images acquired from Landsat 7 ETM+, showing an improvement over results achieved through more basic wavelet algorithms. We also propose the use of spectral concentration during the wavelet packet pan-sharpening process to reduce the dimensionality of the data.
Successful performance of radiological search mission is dependent on effective utilization of mixture of signals. Examples of modalities include, e.g., EO imagery and gamma radiation data, or radiation data collected during multiple events. In addition, elevation data or spatial proximity can be used to enhance the performance of acquisition systems. State of the art techniques in processing and exploitation of complex information manifolds rely on diffusion operators. Our approach involves machine learning techniques based on analysis of joint data- dependent graphs and their associated diffusion kernels. Then, the significant eigenvectors of the derived fused graph Laplace and Schroedinger operators form the new representation, which provides integrated features from the heterogeneous input data. The families of data-dependent Laplace and Schroedinger operators on joint data graphs, shall be integrated by means of appropriately designed fusion metrics. These fused representations are used for target and anomaly detection.
KEYWORDS: Hyperspectral imaging, Data integration, Data fusion, Sensors, Satellite imaging, Earth observing sensors, Satellites, Analytical research, Target detection, Control systems
As new remote sensing modalities emerge, it becomes increasingly important to nd more suitable algorithms
for fusion and integration of dierent data types for the purposes of target/anomaly detection and classication.
Typical techniques that deal with this problem are based on performing detection/classication/segmentation
separately in chosen modalities, and then integrating the resulting outcomes into a more complete picture. In
this paper we provide a broad analysis of a new approach, based on creating fused representations of the multi-
modal data, which then can be subjected to analysis by means of the state-of-the-art classiers or detectors.
In this scenario we shall consider the hyperspectral imagery combined with spatial information. Our approach
involves machine learning techniques based on analysis of joint data-dependent graphs and their associated
diusion kernels. Then, the signicant eigenvectors of the derived fused graph Laplace operator form the new
representation, which provides integrated features from the heterogeneous input data. We compare these fused
approaches with analysis of integrated outputs of spatial and spectral graph methods.
We analyze Schroedinger Eigenmaps - a new semi-supervised manifold learning and recovery technique - for
applications in hyperspectral imagery. This method is based on an implementation of graph Schroedinger
operators with appropriately constructed potentials as carriers of expert/labeled information. In this paper, we
analyze the features of Schroedinger Eigenmaps through analysis of the potential locations and their imapct on
the classication. The imaging modalities which we shall incorporate in our analysis include multispectral and
hyperspectral imagery. For the purpose of constructing ecient methods for building the potentials we refer to
expert ground-truth data, as well as to using automated clustering techniques. We also investigate the role of
dierent sources of the barrier potential locations, and the role they play in the separation of classes.
The topological anomaly detection algorithm (TAD) differs from other anomaly detection algorithms in that
it uses a topological/graph-theoretic model for the image background instead of modeling the image with a
Gaussian normal distribution. In the construction of the model, TAD produces a hard threshold separating
anomalous pixels from background in the image. We build on this feature of TAD by extending the algorithm
so that it gives a measure of the number of anomalous objects, rather than the number of anomalous pixels, in
a hyperspectral image. This is done by identifying, and integrating, clusters of anomalous pixels via a graph
theoretical method combining spatial and spectral information. The method is applied to a cluttered HyMap
image and combines small groups of pixels containing like materials, such as those corresponding to rooftops and
cars, into individual clusters. This improves visualization and interpretation of objects.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.