This PDF file contains the front matter associated with SPIE Proceedings Volume 9842 including the Title Page, Copyright information, Table of Contents, Introduction, and Conference Committee listing.

Visual image tracking involves the estimation of the motion of any desired targets in a surveillance region using a sequence of images. A standard method of isolating moving targets in image tracking uses background subtraction. The standard background subtraction method is often impacted by irrelevant information in the images, which can lead to poor performance in image-based target tracking. In this paper, a B-Spline based image tracking is implemented. The novel method models the background and foreground using the B-Spline method followed by a tracking-by-detection algorithm. The effectiveness of the proposed algorithm is demonstrated.

The challenge of ownship navigation for an airborne platform in the absence of precise navigation information is an important problem. In this paper, the problem is solved using the assumed known GPS locations of landmarks by casting it in a Bayesian state-space framework. It is assumed that no information is available from the navigation sensors. The platform kinematic state is inferred by using a nonlinear filter, such as the extended Kalman filter. The performance is assessed as a function of the density of landmarks and platform manoeuvres in a simulation environment.

We demonstrate the detection and localization performance of a multi-sensor, passive sonar Bayesian tracker for underwater targets emitting narrowband signals in the presence of realistic underwater ambient noise. Our evaluation focuses on recent advances in the formulation of the likelihood function used by the tracker that provide greater robustness in the presence of both realistic environmental noise and imprecise/inaccurate a priori knowledge of the target’s narrowband signal. These improvements enable the tracker to reliably detect and localize narrowband emitters for a broader range of propagation environments, target velocities, and inherent uncertainty in a priori knowledge.

Recent research has developed a novel framework for determining target trackability using the Maximum Likelihood Probabilistic Multi-Hypothesis Tracker (ML-PMHT). This framework allows for the calculation of the PDF of the peak point in the ML-PMHT log-likelihood ratio (LLR) due to clutter as well as the PDF of the peak point in the LLR due to the target. If it is possible to reliably discriminate between the peak target PDF and the peak clutter PDF, then the target is able to be tracked. We expand on this framework by adding a second target and determining the conditions under which both targets can be individually tracked. This work develops the first step toward that goal — it introduces the second target to the framework (an interfering target), and determines how close it can get to the original target before the peak generated by the original target is no longer distinguishable from the peak generated by the interfering target. At this point, the original target will no longer be trackable.

This paper investigates the problem of localizing an unknown number of transient emitters using a network of passive sensors measuring angles of arrival in the presence of missed detections and false alarms. It is assumed that measurements within a certain time window of interest have to be associated before they can be fused to estimate the emitter locations. Two measurement models — either that any target can generate at most one measurement per sensor or that any target can generate several measurements per sensor — are possible within this time window. These two measurement models lead to two different problem formulations: one is an S-D assignment problem and the other is a cardinality selection problem. The S-D assignment problem can be solved by the Lagrangian relaxation algorithm efficiently with a high degree of accuracy when a small number of sensors are used. The sequential m-best 2-D assignment algorithm, which is resistant to the ghosting problem due to the estimation of the emitter signal’s emission time, is developed to solve the problem when the number of sensors becomes large. Simulation results show that the sequential m-best 2-D assignment algorithm is suitable for real time processing with reliable associations and estimates. The cardinality selection formulation models a list of measurements as a Poisson point process and is solved by applying the expectation-maximization (EM) algorithm and an information criterion. The convergence of the EM algorithm to the desired global maximum needs an initialization, which is close to the truth. Localization using passive sensors makes it difficult to obtain such an initial estimate. An assignment-based initialization approach is therefore presented. Simulation studies showed that the EM algorithm based on the assignment initialization is able to estimate the number of targets, target locations and directions with a high degree of accuracy.

This paper introduces the game of reconnaissance blind multi-chess (RBMC) as a paradigm and test bed for understanding and experimenting with autonomous decision making under uncertainty and in particular managing a network of heterogeneous Intelligence, Surveillance and Reconnaissance (ISR) sensors to maintain situational awareness informing tactical and strategic decision making. The intent is for RBMC to serve as a common reference or challenge problem in fusion and resource management of heterogeneous sensor ensembles across diverse mission areas. We have defined a basic rule set and a framework for creating more complex versions, developed a web-based software realization to serve as an experimentation platform, and developed some initial machine intelligence approaches to playing it.

This paper is the fourth in a series aimed at weakening the independence assumptions that are typically presumed in multitarget tracking. Specifically, we assume that, in a multisensory scenario, the sensors are not necessarily independent but, rather, have known correlations (i.e., their joint single-target joint likelihood function is known). From this, we construct a multitarget measurement model for sensors with known correlations. From this model we derive, as an illustrative example, the filtering equations for a probability hypothesis density (PHD) filter for sensors with known correlations. We emphasize the two-sensor case of this filter, for which the measurement-update equations involve a summation over all measurement-to-measurement associations between the two sensors.

Multisensor-multitarget tracking algorithms are typically based on numerous statistical independence assumptions. This paper is the fifth in a series aimed at weakening such assumptions. It addresses the statistics of correlated, simultaneously evolving multitarget populations. The correlation between two multitarget popula-tions is approximately modeled using bivariate i.i.d.c. (independent, identically distributed cluster) distributions. Based on this, a joint tracking filter for such populations is devised, in analogy with the cardinalized probability hypothesis density (CPHD) filter.

Particle filter and Gaussian mixture implementations of random finite set filters have been proposed to tackle the issue of jointly estimating the number of targets and their states. The Gaussian mixture PHD (GM-PHD) filter has a closed-form expression for the PHD for linear and Gaussian target models, and extensions using the extended Kalman filter or unscented Kalman Filter have been developed to allow the GM-PHD filter to accommodate mildly nonlinear dynamics. Errors resulting from linearization or model mismatch are unavoidable. A particle filter implementation of the PHD filter (PF-PHD) is more suitable for nonlinear and non-Gaussian target models. The particle filter implementations are much more computationally expensive and performance can suffer when the proposal distribution is not a good match to the posterior. In this paper, we propose a novel implementation of the PHD filter named the Gaussian particle flow PHD filter (GPF-PHD). It employs a bank of particle flow filters to approximate the PHD; these play the same role as the Gaussian components in the GM-PHD filter but are better suited to non-linear dynamics and measurement equations. Using the particle flow filter allows the GPF-PHD filter to migrate particles to the dense regions of the posterior, which leads to higher efficiency than the PF-PHD. We explore the performance of the new algorithm through numerical simulations.

In sensing applications where multiple sensors observe the same scene, fusing sensor outputs can provide improved results. However, if some of the sensors are providing lower quality outputs, e.g. when one or more sensors has a poor signal-to-noise ratio (SNR) and therefore provides very noisy data, the fused results can be degraded. In this work, a multi-sensor conflict measure is proposed which estimates multi-sensor conflict by representing each sensor output as interval-valued information and examines the sensor output overlaps on all possible n-tuple sensor combinations. The conflict is based on the sizes of the intervals and how many sensors output values lie in these intervals. In this work, conflict is defined in terms of how little the output from multiple sensors overlap. That is, high degrees of overlap mean low sensor conflict, while low degrees of overlap mean high conflict. This work is a preliminary step towards a robust conflict and sensor fusion framework. In addition, a sensor fusion algorithm is proposed based on a weighted sum of sensor outputs, where the weights for each sensor diminish as the conflict measure increases. The proposed methods can be utilized to (1) assess a measure of multi-sensor conflict, and (2) improve sensor output fusion by lessening weighting for sensors with high conflict. Using this measure, a simulated example is given to explain the mechanics of calculating the conflict measure, and stereo camera 3D outputs are analyzed and fused. In the stereo camera case, the sensor output is corrupted by additive impulse noise, DC offset, and Gaussian noise. Impulse noise is common in sensors due to intermittent interference, a DC offset a sensor bias or registration error, and Gaussian noise represents a sensor output with low SNR. The results show that sensor output fusion based on the conflict measure shows improved accuracy over a simple averaging fusion strategy.

Given two legacy exploitation systems, whose performances are known, one might wish to determine if combining these together using some rule would yield a new exploitation system with improved performance. This is the fusion process. Often there are several performance objectives one would consider in this process. We investigate the fusion process based upon multiple performances. This is related to multi-objective optimization, but is different in some aspects. In this paper we pose a multi-performance problem for combining two classifications systems and derive the multi-performance fusion theory. A classification system with M possible output labels will have M(M-1) possible errors. The Receiver Operating Characteristic (ROC) manifold was created to quantify all of these errors. The assumption of independence is usually made to simply the mathematics of combining the individual systems into one system. Boolean rules do not exist for multiple symbols, thus, Boolean-like rules were created that would yield label fusion rules. An M-label system will have M! consistent rules. The formula for the resultant ROC manifold of the fused classification systems which incorporates the individual classification systems previously was derived. For the multi-performance problem we show how the set of permutations of the label set is used to generate all of the consistent rules and how the permutation matrix is incorporated into a single formula for the ROC manifold. Examples will be given that demonstrate how the solution to the multi-performance fusion problem relates to the solution of the single performance fusion problem.

In a recent paper by Mallick and Sindhu, they assert three “problems” with our particle flow theory for Bayes’ rule. Our paper explains why all three assertions are wrong.

We describe many open problems for research in particle flows to compute Bayes’ rule for nonlinear filters, Bayesian decisions and Bayesian learning as well as transport. Particle flow mitigates particle degeneracy, which is the main cause of the curse of dimensionality for particle filters. Particle flow filters are many orders of magnitude faster to compute in real time compared with standard particle filters for the same accuracy for difficult high dimensional problems.

The mathematical similarities between quantum mechanics and stochastic processes has been studied in the literature. Some of the major results are reviewed, such as the relationship between the Fokker-Planck equation and the Schrödinger equation. Also reviewed are more recent results that show the mathematical similarities between quantum many particle systems and concepts in other areas of applied science, such as stochastic Petri nets. Some connections to filtering theory are discussed.

Algorithms developed for the detection of landmines are tasked with discriminating a wide variety of targets in a diverse array of environmental conditions. However, the potential performance of a detection algorithm may be underestimated by evaluating it in batch on a large, diverse dataset. This is because environmental, or in general, contextual, factors may contribute significant variance to the output of a detection algorithm across different contexts. One way to view this is as a problem of miscalibration: within each context, the output scores of a detection algorithm can be seen as miscalibrated relative to the scores produced in the other contexts. As a result of this miscalibration, the observed receiver operating characteristic (ROC) curve for a detector can have a sub-optimal area-under-the-curve (AUC). One solution, then, is to re-calibrate the detector within each context. In this work, we identify multiple sets of contexts in which different landmine detection algorithms exhibit significant output variance and, consequently, miscalibration. We then apply a monotonic calibration strategy that maximizes AUC and demonstrate the gain in observed performance that results when a landmine detection algorithm is properly calibrated within each context.

During the 2015 SPIE DSS conference, nine panelists were invited to highlight the trends and use of context for information fusion. This paper highlights the common issues and trends presented from the panel discussion. The different panelists highlighted methods of filtering methods, data aggregation, and the importance of context for realtime analytics. Using the panelist perspectives, the review organizes the common issues and themes as well areas of future analysis of content and context enrichment from information fusion.

The integration of hard (physical) and soft (meta-physical) contexts in an information fusion system requires the identification of the specific mission oriented goals which it is desired to achieve. Just as most sensors cannot acquire data omnidirectionally, it is not computationally feasible to evaluate all contexts within which acquired data can be understood by an information fusion system. We first define a notional problem consisting of operating and hiding areas and transit routes between them. We then define physical and meta-physical contexts within which data acquired from the observed area can be interpreted and define the piecewise application of context specific transformations to a partition of the global problem of understanding data in context.

Our Multi-INT Data Association Tool (MIDAT) learns patterns of life (POL) of a geographical area from video analyst observations called out in textual reporting. Typical approaches to learning POLs from video make use of computer vision algorithms to extract locations in space and time of various activities. Such approaches are subject to the detection and tracking performance of the video processing algorithms. Numerous examples of human analysts monitoring live video streams annotating or “calling out” relevant entities and activities exist, such as security analysis, crime-scene forensics, news reports, and sports commentary. This user description typically corresponds with textual capture, such as chat. Although the purpose of these text products is primarily to describe events as they happen, organizations typically archive the reports for extended periods. This archive provides a basis to build POLs. Such POLs are useful for diagnosis to assess activities in an area based on historical context, and for consumers of products, who gain an understanding of historical patterns. MIDAT combines natural language processing, multi-hypothesis tracking, and Multi-INT Activity Pattern Learning and Exploitation (MAPLE) technologies in an end-to-end lab prototype that processes textual products produced by video analysts, infers POLs, and highlights anomalies relative to those POLs with links to “tracks" of related activities performed by the same entity. MIDAT technologies perform well, achieving, for example, a 90% F1-value on extracting activities from the textual reports.

Intelligence analysts require automated tools to mine multi-source data, including answering queries, learning patterns of life, and discovering malicious or anomalous activities. Graph mining algorithms have recently attracted significant attention in intelligence community, because the text-derived knowledge can be efficiently represented as graphs of entities and relationships. However, graph mining models are limited to use-cases involving collocated data, and often make restrictive assumptions about the types of patterns that need to be discovered, the relationships between individual sources, and availability of accurate data segmentation. In this paper we present a model to learn the graph patterns from multiple relational data sources, when each source might have only a fragment (or subgraph) of the knowledge that needs to be discovered, and segmentation of data into training or testing instances is not available. Our model is based on distributed collaborative graph learning, and is effective in situations when the data is kept locally and cannot be moved to a centralized location. Our experiments show that proposed collaborative learning achieves learning quality better than aggregated centralized graph learning, and has learning time comparable to traditional distributed learning in which a knowledge of data segmentation is needed.

Various operations such as civil-military co-operation (CIMIC) affairs require orchestration of communications, assets, and actors. A key component includes technology advancements to enable coordination among people and machines the ability to know where things are, who to coordinate with, and open and consistent lines of communication. In this paper, we explore concepts of battle management (BM) to support high-tempo emergency response scenarios such as a disaster action response team (DART). Three concepts highlighted of agile battle management (ABM) include source orchestration (e.g., sensors and domains), battle management language (BML) development (e.g., software and ontologies), and command and control (C2) coordination (e.g., people and visualization); which require correlation and de-confliction. These concepts of ABM support the physical, information, and cognitive domains for efficient command, control, communications, and information (C3I) to synchronize data and people for efficient and effective operations.

Initially designed in the context of the TASS (Total Airport Security System) FP-7 project, the Crowd Simulation platform developed by the Integrated Systems Lab of the Institute of Informatics and Telecommunications at N.C.S.R. Demokritos, has evolved into a complete domain-independent agent-based behavior simulator with an emphasis on crowd behavior and building evacuation simulation. Under continuous development, it reflects an effort to implement a modern, multithreaded, data-oriented simulation engine employing latest state-of-the-art programming technologies and paradigms. It is based on an extensible architecture that separates core services from the individual layers of agent behavior, offering a concrete simulation kernel designed for high-performance and stability. Its primary goal is to deliver an abstract platform to facilitate implementation of several Agent-Based Simulation solutions with applicability in several domains of knowledge, such as: (i) Crowd behavior simulation during [in/out] door evacuation. (ii) Non-Player Character AI for Game-oriented applications and Gamification activities. (iii) Vessel traffic modeling and simulation for Maritime Security and Surveillance applications. (iv) Urban and Highway Traffic and Transportation Simulations. (v) Social Behavior Simulation and Modeling.

wayGoo is a fully functional application whose main functionalities include content geolocation, event scheduling, and indoor navigation. However, significant information about events do not reach users’ attention, either because of the size of this information or because some information comes from real – time data sources. The purpose of this work is to facilitate event management operations by prioritizing the presented events, based on users’ interests using both, static and real – time data. Through the wayGoo interface, users select conceptual topics that are interesting for them. These topics constitute a browsing behavior vector which is used for learning users’ interests implicitly, without being intrusive. Then, the system estimates user preferences and return an events list sorted from the most preferred one to the least. User preferences are modeled via a Naïve Bayesian Network which consists of: a) the ‘decision’ random variable corresponding to users’ decision on attending an event, b) the ‘distance’ random variable, modeled by a linear regression that estimates the probability that the distance between a user and each event destination is not discouraging, ‘ the seat availability’ random variable, modeled by a linear regression, which estimates the probability that the seat availability is encouraging d) and the ‘relevance’ random variable, modeled by a clustering – based collaborative filtering, which determines the relevance of each event users’ interests. Finally, experimental results show that the proposed system contribute essentially to assisting users in browsing and selecting events to attend.

Aerial monitoring applications can be characterized by constantly changing operating conditions, and need for adequate resources to maintain stability, mobility and communication. Heterogeneous aerial sensor network has the ability to move nodes based on application specific goals that can provide three-dimensional sensing, and data distribution using hierarchical communication strategy. In this research paper, swarm based heterogeneous aerial sensor network is deployed on the fly using off-the shelf copters to provide increased sensing accuracy and reliability in comparison networks that require a prior knowledge of the infrastructure. The robustness of swarm approach makes it a suitable algorithm for aerial monitoring environment.

AF3 (Advanced Forest Fire Fighting2) is a European FP7 research project that intends to improve the efficiency of current fire-fighting operations and the protection of human lives, the environment and property by developing innovative technologies to ensure the integration between existing and new systems. To reach this objective, the AF3 project focuses on innovative active and passive countermeasures, early detection and monitoring, integrated crisis management and advanced public information channels. OCULUS Fire is the innovative control and command system developed within AF3 as a monitoring, GIS and Knowledge Extraction System and Visualization Tool. OCULUS Fire includes (a) an interface for real-time updating and reconstructing of maps to enable rerouting based on estimated hazards and risks, (b) processing of GIS dynamic re-construction and mission re-routing, based on the fusion of airborne, satellite, ground and ancillary geolocation data, (c) visualization components for the C2 monitoring system, displaying and managing information arriving from a variety of sources and (d) mission and situational awareness module for OCULUS Fire ground monitoring system being part of an Integrated Crisis Management Information System for ground and ancillary sensors. OCULUS Fire will also process and visualise information from public information channels, social media and also mobile applications by helpful citizens and volunteers. Social networking, community building and crowdsourcing features will enable a higher reliability and less false alarm rates when using such data in the context of safety and security applications.

Complementing the ACI/IATA efforts, the FLYSEC European H2020 Research and Innovation project (http://www.fly-sec.eu/) aims to develop and demonstrate an innovative, integrated and end-to-end airport security process for passengers, enabling a guided and streamlined procedure from the landside to airside and into the boarding gates, and offering for an operationally validated innovative concept for end-to-end aviation security. FLYSEC ambition turns through a well-structured work plan into: (i) innovative processes facilitating risk-based screening; (ii) deployment and integration of new technologies and repurposing existing solutions towards a risk-based Security paradigm shift; (iii) improvement of passenger facilitation and customer service, bringing security as a real service in the airport of tomorrow;(iv) achievement of measurable throughput improvement and a whole new level of Quality of Service; and (v) validation of the results through advanced “in-vitro” simulation and “in-vivo” pilots. On the technical side, FLYSEC achieves its ambitious goals by integrating new technologies on video surveillance, intelligent remote image processing and biometrics combined with big data analysis, open-source intelligence and crowdsourcing. Repurposing existing technologies is also in the FLYSEC objectives, such as mobile application technologies for improved passenger experience and positive boarding applications (i.e. services to facilitate boarding and landside/airside way finding) as well as RFID for carry-on luggage tracking and quick unattended luggage handling. In this paper, the authors will describe the risk based airport security management system which powers FLYSEC intelligence and serves as the backend on top of which FLYSEC’s front end technologies reside for security services management, behaviour and risk analysis.

In the Dempster Shafer context, one can construct new types of information measures based on belief and plausibility functions. These measures differ from those in Shannon’s theory because, in his theory, information measures are based on probability functions. Other types of information measures were discovered by Kampe de Feriet and his colleagues in the French and Italian schools of mathematics. The objective of this paper is to construct a new category of information. I use category theory to construct a general setting in which the various types of information measures are special cases.

A Scene Understanding Challenge Problem was released by AFRL at this conference in 2015 in response to DARPA’s Mathematics, Sensing, Exploitation, and Execution (MSEE) program. We consider a scene understanding system as a generalization of typical sensor exploitation systems where instead of performing a narrowly defined task (e.g., detect, track, classify, etc.), the system can perform general user-defined tasks specified in a query language. That paper¹ laid out the general challenges and methods for developing scene understanding performance models. This is an enormously challenging problem, so now AFRL is illustrating the methods with a baseline system primarily developed by the University of California, Los Angeles (UCLA) during the MSEE program. This system will be publicly available for others to utilize, compare, and contrast with related methods. This paper will further explain and provide insights into the challenges, illustrating them with examples from a publicly available data set. Our intent is that these tools will relieve the requirement for developing an entire system and enable progress to occur by focusing on individual elements of the system. Finally, we will provide details as to how interested researchers may obtain the system and the data.

This paper describes a novel sensor resolution assessment chart and procedure based on the Landolt C. This automated resolution assessment procedure does not rely on human vision as the primary means of interpreting resolution chart results. Seven sensors across four different spectral bands and several geometric resolutions were assessed a total of five times each to determine the repeatability (confidence interval) for this automated procedure. The results are presented and compared with previous studies.

Visible spectrum face detection algorithms perform pretty reliably under controlled lighting conditions. However, variations in illumination and application of cosmetics can distort the features used by common face detectors, thereby degrade their detection performance. Thermal and polarimetric thermal facial imaging are relatively invariant to illumination and robust to the application of makeup, due to their measurement of emitted radiation instead of reflected light signals. The objective of this work is to evaluate a government off-the-shelf wavelet based naïve-Bayes face detection algorithm and a commercial off-the-shelf Viola-Jones cascade face detection algorithm on face imagery acquired in different spectral bands. New classifiers were trained using the Viola-Jones cascade object detection framework with preprocessed facial imagery. Preprocessing using Difference of Gaussians (DoG) filtering reduces the modality gap between facial signatures across the different spectral bands, thus enabling more correlated histogram of oriented gradients (HOG) features to be extracted from the preprocessed thermal and visible face images. Since the availability of training data is much more limited in the thermal spectrum than in the visible spectrum, it is not feasible to train a robust multi-modal face detector using thermal imagery alone. A large training dataset was constituted with DoG filtered visible and thermal imagery, which was subsequently used to generate a custom trained Viola-Jones detector. A 40% increase in face detection rate was achieved on a testing dataset, as compared to the performance of a pre-trained/baseline face detector. Insights gained in this research are valuable in the development of more robust multi-modal face detectors.

In many military and homeland security persistent surveillance applications, accurate detection of different skin colors in varying observability and illumination conditions is a valuable capability for video analytics. One of those applications is In-Vehicle Group Activity (IVGA) recognition, in which significant changes in observability and illumination may occur during the course of a specific human group activity of interest. Most of the existing skin color detection algorithms, however, are unable to perform satisfactorily in confined operational spaces with partial observability and occultation, as well as under diverse and changing levels of illumination intensity, reflection, and diffraction. In this paper, we investigate the salient features of ten popular color spaces for skin subspace color modeling. More specifically, we examine the advantages and disadvantages of each of these color spaces, as well as the stability and suitability of their features in differentiating skin colors under various illumination conditions. The salient features of different color subspaces are methodically discussed and graphically presented. Furthermore, we present robust and adaptive algorithms for skin color detection based on this analysis. Through examples, we demonstrate the efficiency and effectiveness of these new color skin detection algorithms and discuss their applicability for skin detection in IVGA recognition applications.

Human activity detection and recognition capabilities have broad applications for military and homeland security. These tasks are very complicated, however, especially when multiple persons are performing concurrent activities in confined spaces that impose significant obstruction, occultation, and observability uncertainty. In this paper, our primary contribution is to present a dedicated taxonomy and kinematic ontology that are developed for in-vehicle group human activities (IVGA). Secondly, we describe a set of hand-observable patterns that represents certain IVGA examples. Thirdly, we propose two classifiers for hand gesture recognition and compare their performance individually and jointly. Finally, we present a variant of Hidden Markov Model for Bayesian tracking, recognition, and annotation of hand motions, which enables spatiotemporal inference to human group activity perception and understanding. To validate our approach, synthetic (graphical data from virtual environment) and real physical environment video imagery are employed to verify the performance of these hand gesture classifiers, while measuring their efficiency and effectiveness based on the proposed Hidden Markov Model for tracking and interpreting dynamic spatiotemporal IVGA scenarios.

Human group activity recognition is a very complex and challenging task, especially for Partially Observable Group Activities (POGA) that occur in confined spaces with limited visual observability and often under severe occultation. In this paper, we present IRIS Virtual Environment Simulation Model (VESM) for the modeling and simulation of dynamic POGA. More specifically, we address sensor-based modeling and simulation of a specific category of POGA, called In-Vehicle Group Activities (IVGA). In VESM, human-alike animated characters, called humanoids, are employed to simulate complex in-vehicle group activities within the confined space of a modeled vehicle. Each articulated humanoid is kinematically modeled with comparable physical attributes and appearances that are linkable to its human counterpart. Each humanoid exhibits harmonious full-body motion - simulating human-like gestures and postures, facial impressions, and hands motions for coordinated dexterity. VESM facilitates the creation of interactive scenarios consisting of multiple humanoids with different personalities and intentions, which are capable of performing complicated human activities within the confined space inside a typical vehicle. In this paper, we demonstrate the efficiency and effectiveness of VESM in terms of its capabilities to seamlessly generate time-synchronized, multi-source, and correlated imagery datasets of IVGA, which are useful for the training and testing of multi-source full-motion video processing and annotation. Furthermore, we demonstrate full-motion video processing of such simulated scenarios under different operational contextual constraints.

Structural fires continue to pose a great threat towards human life and property. Due to the complexity and non-deterministic characteristics of a building fire disaster, it is not a straightforward task to assess the effectiveness of fire protection measures embedded in the building design, planned evacuation strategies and potential modes of response for mitigating the fire’s consequences. Additionally, there is a lack of means that realistically and accurately recreate the conditions of building fire disasters for the purpose of training personnel in order to be sufficiently prepared when vis-a-vis with such an environment. The propagation of fire within a building, the diffusion of its volatile products, the behavior of the occupants and the sustained injuries not only exhibit non-linear behaviors as individual phenomena, but are also intertwined in a web of co-dependencies. The PYRONES system has been developed to address all these aspects through a comprehensive approach that relies on accurate and realistic computer simulations of the individual phenomena and their interactions. PYRONES offers innovative tools and services to strategically targeted niches in two market domains. In the domain of building design and engineering, PYRONES is seamlessly integrated within existing engineering Building Information Modelling (BIM) workflows and serves as a building performance assessment platform, able to evaluate fire protection systems. On another front, PYRONES penetrates the building security management market, serving as a holistic training platform for specialists in evacuation strategy planning, firefighters and first responders, both at a Command and Control and at an individual trainee level.

We present a system for people counting and re-identification. It can be used by transit and homeland security agencies. Under FTA SBIR program, we have developed a preliminary system for transit passenger counting and re-identification using a laser scanner and video camera. The laser scanner is used to identify the locations of passenger’s head and shoulder in an image, a challenging task in crowed environment. It can also estimate the passenger height without prior calibration. Various color models have been applied to form color signatures. Finally, using a statistical fusion and classification scheme, passengers are counted and re-identified.

wayGoo² is a platform for Geolocating and Managing indoor and outdoor spaces and content with multidimensional indoor and outdoor Navigation and Guidance. Its main components are a Geographic Information System, a back-end server, front-end applications and a web-based Content Management System (CMS). It constitutes a fully integrated 2D/3D space and content management system that creates a repository that consists of a database, content components and administrative data. wayGoo can connect to any third party database and event management data-source. The platform is secure as the data is only available through a Restful web service using https security protocol in conjunction with an API key used for authentication. To enhance users experience, wayGoo makes the content available by extracting components out of the repository and constructing targeted applications. The wayGoo platform supports geo-referencing of indoor and outdoor information and use of metadata. It also allows the use of existing information such as maps and databases. The platform enables planning through integration of content that is connected either spatially, temporally or contextually, and provides immediate access to all spatial data through interfaces and interactive 2D and 3D representations. wayGoo constitutes a mean to document and preserve assets through computerized techniques and provides a system that enhances the protection of your space, people and guests when combined with wayGoo notification and alert system. It constitutes a strong marketing tool providing staff and visitors with an immersive tool for navigation in indoor spaces and allowing users to organize their agenda and to discover events through wayGoo event scheduler and recommendation system. Furthermore, the wayGoo platform can be used in Security applications and event management, e.g. CBRNE incidents, man-made and natural disasters, etc., to document and geolocate information and sensor data (off line and real time) on one end, and offer navigation capabilities in indoor and outdoor spaces. Furthermore, the wayGoo platform can be used for the creation of immersive environments and experiences in conjunction with VR/AR (Virtual and Augmented Reality) technologies.

Location-based and navigation services are really needed to help visitors and audience of big events, complex buildings, shopping malls, airports and large companies. However, the lack of GPS and proper mapping indoors usually renders location-based applications and services useless or simply not applicable in such environments. SYNAISTHISI introduces a mobile application for smartphones which offers navigation capabilities outside and inside buildings and through multiple floor levels. The application comes together with a suite of helpful services, including personalized recommendations, visit/event management and a helpful search functionality in order to navigate to a specific location, event or person. As the user finds his way towards his destination, NFC-enabled checkpoints and bluetooth beacons assist him, while offering re-routing, check-in/out capabilities and useful information about ongoing meetings and nearby events. The application is supported by a back-end GIS system which can provide a broad and clear view to event organizers, campus managers and field personnel for purposes of event logistics, safety and security. SYNAISTHISI system comes with plenty competitive advantages including (a) Seamless Navigation as users move between outdoor and indoor areas and different floor levels by using innovative routing algorithms, (b) connection to and powered by IoT platform, for localization and real-time information feedback, (c) dynamic personalized recommendations based on user profile, location and real-time information provided by the IoT platform and (d) Indoor localization without the need for expensive infrastructure and installations.

The scope of the present study is to research the dynamics that determine the commission of crimes in the US society. Our study is part of a model we are developing to understand urban crime dynamics and to enhance citizens’ “perception of security” in large urban environments. The main targets of our research are to highlight dependence of crime rates on certain social and economic factors and basic elements of state anticrime policies. In conducting our research, we use as guides previous relevant studies on crime dependence, that have been performed with similar quantitative analyses in mind, regarding the dependence of crime on certain social and economic factors using statistics and econometric modelling. Our first approach consists of conceptual state space dynamic cross-sectional econometric models that incorporate a feedback loop that describes crime as a feedback process. In order to define dynamically the model variables, we use statistical analysis on crime records and on records about social and economic conditions and policing characteristics (like police force and policing results – crime arrests), to determine their influence as independent variables on crime, as the dependent variable of our model. The econometric models we apply in this first approach are an exponential log linear model and a logit model. In a second approach, we try to study the evolvement of violent crime through time in the US, independently as an autonomous social phenomenon, using autoregressive and moving average time-series econometric models. Our findings show that there are certain social and economic characteristics that affect the formation of crime rates in the US, either positively or negatively. Furthermore, the results of our time-series econometric modelling show that violent crime, viewed solely and independently as a social phenomenon, correlates with previous years crime rates and depends on the social and economic environment’s conditions during previous years.

Target detection is one of the most important topics for military or civilian applications. In order to address such detection tasks, hyperspectral imaging sensors provide useful images data containing both spatial and spectral information. Target detection has various challenging scenarios for hyperspectral images. To overcome these challenges, covariance descriptor presents many advantages. Detection capability of the conventional covariance descriptor technique can be improved by fusion methods. In this paper, hyperspectral bands are clustered according to inter-bands correlation. Target detection is then realized by fusion of covariance descriptor results based on the band clusters. The proposed combination technique is denoted Covariance Descriptor Fusion (CDF). The efficiency of the CDF is evaluated by applying to hyperspectral imagery to detect man-made objects. The obtained results show that the CDF presents better performance than the conventional covariance descriptor.

A sector-based angular scanning system intended to identify and spatially locate relatively small objects scattered over a large terrain is described in this paper. The system is modeled as a planar surface on the horizontal (XY) plane, with an acousto-optic Bragg cell on board an unmanned aerial vehicle (UAV) operating in the XZ plane. The Bragg cell is excited by a chirped RF signal with a designed frequency ramp. As the scanning beam reflects off the horizontal surface, a detector placed strategically at a suitable altitude (in the analysis shown to be on board the UAV itself) picks up the reflected wave and thereafter evaluates the refractive index of the material at the location using the Fresnel reflection coefficient. For large area coverage, the UAV makes alternate 180-degree turns at the end of each row-scan, thereby after several row scans, a practical surface area is covered. The usefulness and limitations of this scanning method are discussed.

This paper is devoted to addressing the synchronization, and detection of random binary data exposed to inherent channel variations existing in Free Space Optical (FSO) communication systems. This task is achieved by utilizing the identical synchronization methodology of Lorenz chaotic communication system, and its synergetic interaction in adversities imposed by the FSO channel. Moreover, the Lorenz system has been analyzed, and revealed to induce Stochastic Resonance (SR) once exposed to Additive White Gaussian Noise (AWGN). In particular, the resiliency of the Lorenz chaotic system, in light of channel adversities, has been attributed to the success of the proposed communication system. Furthermore, this paper advocates the use of Haar wavelet transform for enhanced detection capability of the proposed chaotic communication system, which utilizes Chaotic Parameter Modulation (CPM) technique for means of transmission.

This paper considers the ubiquitous problem of estimating the state (e.g., position) of an object based on a series of noisy measurements. The standard approach is to formulate this problem as one of measuring the state (or a function of the state) corrupted by additive Gaussian noise. This model assumes both (i) the sensor provides a measurement of the true target (or, alternatively, a separate signal processing step has eliminated false alarms), and (ii) The error source in the measurement is accurately described by a Gaussian model. In reality, however, sensor measurement are often formed on a grid of pixels – e.g., Ground Moving Target Indication (GMTI) measurements are formed for a discrete set of (angle, range, velocity) voxels, and EO imagery is made on (x, y) grids. When a target is present in a pixel, therefore, uncertainty is not Gaussian (instead it is a boxcar function) and unbiased estimation is not generally possible as the location of the target within the pixel defines the bias of the estimator. It turns out that this small modification to the measurement model makes traditional bounding approaches not applicable. This paper discusses pixelated sensing in more detail and derives the minimum mean squared error (MMSE) bound for estimation in the pixelated scenario. We then use this error calculation to investigate the utility of using non-thresholded measurements.

As biometrics become increasingly pervasive, consumer electronics are reaping the benefits of improved authentication methods. Leveraging the physical characteristics of a user reduces the burden of setting and remembering complex passwords, while enabling stronger security. Multi-factor systems lend further credence to this model, increasing security via multiple passive data points. In recent years, brainwaves have been shown to be another feasible source for biometric authentication. Physically unique to an individual in certain circumstances, the signals can also be changed by the user at will, making them more robust than static physical characteristics. No paradigm is impervious however, and even well-established medical technologies have deficiencies. In this work, a system for biometric authentication via brainwaves is constructed with electroencephalography (EEG). The efficacy of EEG biometrics via existing consumer electronics is evaluated, and vulnerabilities of such a system are enumerated. Impersonation attacks are performed to expose the extent to which the system is vulnerable. Finally, a multimodal system combining EEG with additional factors is recommended and outlined.

A novel hyperspectral target detection technique based on Fukunaga-Koontz transform (FKT) is presented. FKT offers significant properties for feature selection and ordering. However, it can only be used to solve multi-pattern classification problems. Target detection may be considered as a two-class classification problem, i.e., target versus background clutter. Nevertheless, background clutter typically contains different types of materials. That’s why; target detection techniques are different than classification methods by way of modeling clutter. To avoid the modeling of the background clutter, we have improved one-class FKT (OC-FKT) for target detection. The statistical properties of target training samples are used to define tunnel-like boundary of the target class. Non-target samples are then created synthetically as to be outside of the boundary. Thus, only limited target samples become adequate for training of FKT. The hyperspectral image experiments confirm that the proposed OC-FKT technique provides an effective means for target detection.

In this paper, a comprehensive comparison is made of the following sigma-point Kalman filters: unscented Kalman filter (UKF), cubature Kalman filter (CKF), and the central difference Kalman filter (CDKF). A simulation based on a complex maneuvering road (an s-path) is used as a benchmark problem. This paper studies the response, stability, robustness, convergence, and computational complexity of the filters. Future work will look at implementing the methods on a robot built for experimentation.

Multiple target tracking (MTT) is a challenging task that aims to estimate the number of targets and their states in the presence of process noise, measurement noise and data association uncertainty. This paper considers a special MTT problem characterized by additional complexity. In this problem, multiple targets are launched simultaneously in nearby locations at the same speed with slightly different directions. As the distances be-tween the initial locations of these targets are smaller than the resolution of the sensor, this results in merged measurements, i.e., unresolved tracks at the very beginning. To deal with this problem, the recently proposed Multi-Bernoulli (MB) filter is applied. Using a model for the merged measurements, simulation results with 2-D Cartesian measurements in an optical sensor’s focal plane in the presence of clutter show that the initially unresolved tracks become resolved with MB filtering a few time steps after the measurements become resolved. Thus, the MB filter is capable of keeping track of the number of targets and their corresponding states when they are initially unresolved.

Attitude determination is one of the most important subsystems in spacecraft, satellite, or scientific balloon mission s, since it can be combined with actuators to provide rate stabilization and pointing accuracy for payloads. In this paper, a low-cost attitude determination system with a precision in the order of arc-seconds that uses low-cost commercial sensors is presented including a set of uncorrelated MEMS gyroscopes, two clinometers, and a magnetometer in a hierarchical manner. The faster and less precise sensors are updated by the slower, but more precise ones through an Extended Kalman Filter (EKF)-based data fusion algorithm. A revision of the EKF algorithm fundamentals and its implementation to the current application, are presented along with an analysis of sensors noise. Finally, the results from the data fusion algorithm implementation are discussed in detail.

This paper proposes a new approach on the Holter monitor by creating a portable Electrocardiogram (ECG) Holter monitor that will alert the user by detecting abnormal heart beats using a digital signal processing software. The alarm will be triggered when the patient experiences arrhythmias such as bradycardia and tachycardia. The equipment is simple, comfortable and small in size that fit in the hand. It can be used at any time and any moment by placing three leads to the person’s chest which is connected to an electronic circuit. The ECG data will be transmitted via Bluetooth to the memory of a selected mobile phone using an application that will store the collected data for up to 24 hrs. The arrhythmia is identified by comparing the reference signals with the user’s signal. The diagnostic results demonstrate that the ECG Holter monitor alerts the user when an arrhythmia is detected thru the Holter monitor and mobile application.

United States Special Operations Command (SOCOM) recently published a white paper describing the “Gray Zone”, security challenges characterized by “ambiguity about the nature of the conflict, opacity of the parties involved…competitive interactions among and within state and non-state actors that fall between the traditional war and peace duality.”¹ Ambiguity and related uncertainty about actors, situations, relationships, and intent require new approaches to information collection, processing and fusion. General Votel, the current SOCOM commander, during a recent speech on “Operating in the Gray Zone” emphasized that it would be important to get left of the next crises and stated emphatically, “to do that we must understand the Human Domain.”² This understanding of the human domain must come from making meaning based on different perspectives, including the “emic” or first person/participant and “etic” or third person/observer perspectives. Much of the information currently collected and processed is etic. Incorporation and fusion with the emic perspective enables forecasting of behaviors/events and provides context for etic information (e.g., video).³ Gray zone challenges are perspective-dependent; for example, the conflict in Ukraine is interpreted quite differently by Russia, the US and Ukraine. Russia views it as war, necessitating aggressive action, the US views it as a security issue best dealt with by economic sanctions and diplomacy and the Ukraine views it as a threat to its sovereignty.⁴ General Otto in the Air Force ISR 2023 vision document stated that Air Force ISR is needed to anticipate strategic surprise.⁵ Anticipatory analysis enabling getting left of a crisis inherently requires a greater focus on information sources that elucidate the human environment as well as new methods that elucidate not only the “who’s” and “what’s”, but the “how’s and “why’s,” extracting features and/or patterns and subtle cues useful for forecasting behaviors and events; for example discourse patterns related to social identity and integrative complexity.⁶ AFRL has been conducting research to enable analysts to understand the “emic” perspective based on discourse analysis methods and/or text analytics.⁷ Previous results demonstrated the value of fusion of emic and etic information in terms of improved accuracy (from 39% to 86%) in forecasting violent events.8 This paper will describe new work to extend this to anticipatory analysis in the gray zone.