Two different visual descriptions provided by two image sensors (radar and infrared camera) contain information of the same scene. We want to associate them, using different methods of fusion, in order to improve our knowledge of the scene. Two approaches are described in this paper: navigation and recognition. In the first approach, the radar is the predominant sensor and we use cartographic information of the area to guide the fusion process. In the second approach, we find regions of interest in the radar image that are used to extract features in the infrared image. To experiment our algorithm, we are using a PtSi infrared camera (3-5jtm) with a 512*5 12 matrix and a millimeterwave radar, that are looking at the same area from an airplane, to detect objects like buildings, roads, fields ... . It is the basis of further developments within an expert system including more complex notions of image processing objects.

Two algorithms, maximum a posteriori (MAP) estimation and the Dempster-shafer evidential reasoning technique, for multi-sensor or multi-spectral image data fusion for image segmentation are presented and cornpared. Regions of the images observed by each sensor are modeled as noncausal Gauss Markov random fields (GMRF) and labeled images are assumed to follow a Gibbs distribution. In the Bayesian MAP approach, we use an independent opinion pool for data fusion and a deterministic relaxation to obtain the MAP solution. In practice, the Bayesian approach is too restrictive and a likelihood represented by a point probability value is usually an overstatement of what is actually known. In the Dempster-Shafer approach, we adopt Dempster's rule of combination for data fusion, using belief intervals and ignorance to represent confidence in a particular labeling and we present a new deterministic relaxation scheme that updates the belief intervals. Results obtained from mosaic images of real textures in a three hypothetical sensors problem are presented and the two algorithms are quantitatively compared.

An algorithm to combine multiple data with dependency information in a distributed sensor system is proposed. Three types of dependency information are considered: independency, maximum, and minimum dependency. A modified Schienman technique to determine nonoverlapping subhypotheses is presented.

The association or "fusion" of multiple-sensor reports allows the generation of a highly accurate description of the environment by enabling efficient compression and processing of otherwise unwieldy quantities of data. Assuming that the observations from each sensor are aligned in feature space and in time, this association procedure may be executed on the basis of how well each sensor's vectors of observations match previously fused tracks. Unfortunately, distance-based algorithms alone do not suffice in those situations where match-assignments are not of an obvious nature (e.g., high target density or high false alarm rate scenarios). Our proposed approach is based on recognizing that, together, the sensors' observations and the fused tracks span a vector subspace whose dimensionality and singularity characteristics can be used to determine the total number of targets appearing across sensors. A properly constrained transformation can then be found which aligns the subspaces spanned individually by the observations and by the fused tracks, yielding the relationship existing between both sets of vectors ("Procrustes Problem"). The global nature of this approach thus enables fusing closely-spaced targets by treating them--in a manner analogous to PDA/JPDA algorithms - as clusters across sensors. Since our particular version of the Procrustes Problem consists basically of a minimization in the Total Least Squares sense, the resulting transformations associate both observations-to-tracks and tracks-to--observations. This means that the number of tracks being updated will increase or decrease depending on the number of targets present, automatically initiating or deleting "fused" tracks as required, without the need of ancillary procedures. In addition, it is implicitly assumed that both the tracker filters' target trajectory models and the sensors' observations are "noisy", yielding an algorithm robust even against maneuvering targets. Finally, owing to the fact that Procrustes Association yields the optimal linear associator, the combined sensor and fused track information minimizes tracking Kalman Filter residuals, hence providing very accurate track updates.

An assessment of numerous activities in the field of multisensor target recognition reveals several trends and conditions which are cause for concern. .These concerns are analyzed in terms of their potential impact on the ultimate employment of automatic target recognition in military systems. Suggestions for additional investigation and guidance for current activities are presented with respect to some of the identified concerns.

Long term diffraction limited observations in the solar system require line of sight accuracies in the 0.01 arcsec regime and closed loop on object tracking capabilities. To overcome most of the problems related to the verification of a reference system utilizing high precision gyros, a control system based on image motion control and a pure optical inertial reference system was investigated. The performance of the proposed control system was evaluated via nonlinear simulations.

Typically, air-to-air missiles built to this day use either radar or infrared guidance to steer a missile on to the target. This paper presents the development of a multisensor seeker for medium range air-to-air missiles which would contain active infrared (IR) and radio frequency (RF) modes, as well as passive IR and RF modes. The principal technology issues identified, and techniques proposed include a cylindrically shaped, frequency scanned, conformal RF antenna array and a gimballed heterodyne/direct double detector IR sensor in an array configuration. The IR sensor consists of a dual active/passive imaging system mounted on a gimballed platform, substituted at the focal plane of the optical system. Alternative design options for both RF and IR sensors are also suggested.

This paper demonstrates the advantages of applying raultisensor data fusion techniques for the mine detection and classification problem. It has long been recognized that no single sensor technique is adequate for the detection and classification of the wide spectrum of anti-personnel, anti-vehicle and anti-tank mines currently deployed under an even wider spectrum of battle and geological conditions. The experimental results have indicated that mine detection and classification performance can be greatly enhanced by using an array of complementary sensors whose outputs are fused to extract information not otherwise available from a single sensor.

The development of a reliable Automatic Target Recognition (ATE) system is considered a very critical and challenging problem. Existing ATE Systems have inherent limitations in terms of recognition performance and the ability to learn and adapt. Artificial Intelligence Techniques have the potential to improve the performance of ATh Systems. In this paper, we presented a novel Knowledge-Engineering tool, termed, the Automatic Reasoning Process (ARP) , that can be used to automatically develop and maintain a Knowledge-Base (K-B) for the ATR Systems. In its learning mode, the ARP utilizes Learning samples to automatically develop the ATR K-B, which consists of minimum size sets of necessary and sufficient conditions for each target class. In its operational mode, the ARP infers the target class from sensor data using the ATh K-B System. The ARP also has the capability to reason under uncertainty, and can support both statistical and model-based approaches for ATR development. The capabilities of the ARP are compared and contrasted to those of another Knowledge-Engineering tool, termed, the Automatic Rule Induction (ARI) which is based on maximizing the mutual information. The AR? has been implemented in LISP on a VAX-GPX workstation.

This paper describes an approach for intruder detection by a remote surveillance system. Emphasis is placed on the development of multispectral vision techniques for the extraction of information from a noisy and cluttered environment. This approach uses adaptive detection for operation under changing illumination and thermal environments. The system is initialized with an operator-guided segmentation to partition the scene into regions of similar noise characteristics and processing priorities. Images from a color TV and a FLIR are registered electronically and a common segmentation is used for both. Detection processing in corresponding regions of the two sensors' images are closely coupled. A system testbed is described and several processing sequences are presented which illustrate the approach.

The following problem is studied: to describe the performance of a network of sensors in effecting a binary classification of perceived targets, perhaps comprising several distinct classes (multiple decoy types, for example), by independent observations. Sensor performance is modeled either by target response densities or by receiver operating characteristic curves. We show how a joint ROC curve can be constructed when either of these models is used and illustrate some of the ideas by examples. A key idea is a new sensor model, called herein "neoclassical", that yields a one dimensional equivalent sensor model for a multisensor network.

This paper compares preference voting techniques for multi-sensor decision support. The domain we are concerned with is enabling integration of visual information from diverse kinds of sensors. We define conditions imposed by: a) multi-source information fusion tasks, and b) models of multi-source decision-making processes. The results are directed toward two key problems: a) facilitating group decisions via computer support, and b) selection of related items from an image data base. We describe three methods for combining multiple assessments of image content from different sources. Two of these make use of diversity of knowledge possessed by contributers to the group decision. We assume a heterogeneous voter pool (individuals or programs), and also different credibilities for their inputs to the group decision. We present simulation results for the three decision-making methods.

A conducting PRIZ device is fabricated and shown to exhibit the unique image processing features of edge detection, directional spatial filtering, and dynamic image selection. The active element is single crystal bismuth silicon oxide (Bi12SiO20 or BSO) . Auger electron spectroscopy (AES) and neutron activation analysis (NAA) techniques were applied to BSO to identify the impurities which contribute to the device operation.

The problem of associating data in a domain with noisy sensor inputs is of considerable importance in a wide variety of problem areas. Data association algorithms provide an approach for automatically correlating and combining incoming sensor data. A number of association algorithms have been developed; however, evaluating the effectiveness of these algorithms is difficult because traditional evaluation methods fail to provide meaningful meansures of relative merit. These traditional measures are troublesome because the type I and type II errors upon which they are based lose all meaning after reports are combined in a data base. This paper describes a test bed which uses an alternative approach for measuring the performance of association algorithms. Like the traditional measures, the approach described here requires the use of simulated sensor data. The evaluation procedure is based on a measure of the distance between a baseline representation and the representation produced by the association algorithm at some time instant. Two choices for this baseline representation are listed and scores are defmed between these baselines and an algorithm's representation. A description of the test bed architecture which implements this evaluation procedure is provided, as well as, sample outputs from performing algorithm evaluations in the test bed.

Optimality of decisions in multi-sensor environments calls for processing and management of large volumes of data with differing resolutions, varied noise/clutter background conditions and rapidly changing environmental scenarios. This represents a potentially challenging task especially in view of the real-time constraints imposed by environments such as those encountered in defense applications. The study starts with a discussion of how the fusion process can be conceived at different levels. This is followed by a presentation of the alternative categorizations of the multisensor environment based on the characteristics of the knowledge available therein. Different paradigms corresponding to these categorizations are presented starting with traditional paradigms for learning in completely known environments. But these do not always meet the challenge posed by real world mult.isensor environments. This calls for more adaptive learning strategies to efficiently tackle the dynamic nature of the environment. Accordingly, a specirum of paradigms are presented which are designed to aid synergistic learning and reliable decision making and serve as a guide to the conceptual design of information processing systems in multi-sensor environments. In addition, several avenues for further research and developmental efforts are identified during the presentation of these paradigms.

This paper describes an approach for intruder detection by a remote surveillance system. Emphasis is placed on the development of multispectral vision techniques for the extraction of information from a noisy and cluttered environment. This approach uses adaptive detection for operation under changing illumination and thermal environments. The system is initialized with an operator-guided segmentation to partition the scene into regions of similar noise characteristics and processing priorities. Images from a color TV and a FLIR are registered electronically and a common segmentation is used for both. Detection processing in corresponding regions of the two sensors' images are closely coupled. A system testbed is described and several processing sequences are presented which illustrate the approach.

A method for fabrication of novel thin-filrn continuous dynode electron multipliers is described. We have shown the feasibility of crucial manufacturing steps, including anisotropic dry etching of substrates into photolithographically-defined arrays of high-aspect-ratio channels, and the formation of thin-film continuous dynodes by chemical vapor deposition. We discuss potential performance and design advantages of this advanced technology microchannel plate (AT-MCP) over the conven tional reduced lead silicate glass inicrochannel plate (RLSG-'MCP) and implications for new applications.

An assessment of numerous activities in the field of multisensor target recognition reveals several trends and conditions which are cause for concern. .These concerns are analyzed in terms of their potential impact on the ultimate employment of automatic target recognition in military systems. Suggestions for additional investigation and guidance for current activities are presented with respect to some of the identified concerns.

Smart weapons are expected to autonomously seek out and attack a wide variety of strategic and tactical targets. These targets are assumed to be hidden, camouflaged, covered with foliage and at locations not accurately known in advance of the mission. To be effective, however, a sensor suite must be provided that is compatible with the constraints imposed by the smart weapons in terms of size, weight, power and cost limitations. The sensor suite must provide for navigation, target detection, acquisition, identification and verification under adverse (all) weather conditions. This study surveys the current and projected future status of a family of smart weapon sensors potentially capable of performing all the sensing tasks required by these weapons.

We consider the problem of target discrimination based on measurements taken by two sensors, with each sensor measuring a single continuous quantity. We analyze several fusion schemes. These differ in how much data compression the individual sensors make before sending their information to the fusion center where the final decision is made, and in how that fmal decision is made. A comparison of the performance of the various fusion schemes is done by comparing their ROCs (Receiver Operating Characteristics).

Automatic Target Recognition algorithms have been developed with limited success over the last few years. The processing to extract military targets from background clutter has difficulty under noisy, real-world conditions. Fusion of data from different wavelength sensors has been one approach to improve performance. The underlying theory is that signature data from different areas of the electro-magnetic spectrum will be complementary and clutter is frequency dependent. Recent work based on both statistical classification, and feature analysis in the thermal infrared and millimeter wave spectra, has shown interesting trends. We will provide a description of the IR/MMW target classification algorithms, the fusion architecture we employed, and processes used to search for the optimum features. Two distinct search and detect schemes were tested with different results. Data acquisition and reduction issues which affect algorithm experiments will also be discussed. A software based algorithm development test-bed was built at Textron to implement the multispectral targeting experiments. The effect of a modular, programmable test-bed on such experiments is to increase productivity and allow multivariate evaluatio ns.