Hyperspectral (HS) data contains spectral response information that provides detailed chemical, moisture, and other descriptions of constituent parts of an item. These new sensor data are useful in USDA product inspection and in automatic target recognition (ATR) applications. However, such data introduces problems such as the curse of dimensionality, the need to reduce the number of features used to accommodate realistic small training set sizes, and the need to employ discriminatory features and still achieve good generalization (comparable training and test set performance). HS produces high-dimensional data; this is characterized by a training set size (Ni) per class that is less than the number of input features (HS (lambda) bands). A new high-dimensional generalized discriminant (HDGD) feature extraction algorithm and a new high-dimensional branch and bound (HDBB) feature selection are described and compared to other feature reduction methods for two HS product inspection application. Cross-validation methods, not using the test set, select algorithm parameters.

During the last decade, there has been significant progress toward a supervised autonomous robotic capability for remotely controlled scientific exploration of planetary surfaces. While planetary exploration potentially encompasses many elements ranging from orbital remote sensing to subsurface drilling, the surface robotics element is particularly important to advancing in situ science objectives. Surface activities include a direct characterization of geology, mineralogy, atmosphere and other descriptors of current and historical planetary processes-and ultimately-the return of pristine samples to Earth for detailed analysis. Toward these ends, we have conducted a broad program of research on robotic systems for scientific exploration of the Mars surface, with minimal remote intervention. The goal is to enable high productivity semi-autonomous science operations where available mission time is concentrated on robotic operations, rather than up-and-down-link delays. Results of our work include prototypes for landed manipulators, long-ranging science rovers, sampling/sample return mobility systems, and more recently, terrain-adaptive reconfigurable/modular robots and closely cooperating multiple rover systems. The last of these are intended to facilitate deployment of planetary robotic outposts for an eventual human-robot sustained scientific presence. We overview our progress in these related areas of planetary robotics R&D, spanning 1995-to-present.

Pattern matching is an important task in industrial applications of computer vision. A variety of different approaches to the pattern matching problem has been proposed in the literature. In this paper we provide an experimental evaluation of three different matching algorithms, namely a feature consensus matching algorithm, an interpretation tree based algorithm and a commercially available pattern matching tool. The systems were chosen to cover a range of matching techniques as well as a large field of possible applications. Both artificially created test images and image sequences of real-world matching problems have been applied to indicate the range of feasible applications to any of the three matching system. The algorithms are investigated with respect to matching speed, accuracy and robustness.

We study an object recognition system where Bayesian inference is used for estimating the probability distribution of matching object locations on an image. The representation of the object contains two parts: the likelihood part that defines the probability of perceiving a given (gray scale) image corresponding to the matched object detail, and the prior part that defines the probability of variation of the object, including elastic distortions and interclass variations. The application we are studying is related to recognition of faces and recovering the shape of human heads. The prior consists of covariance model for the face, which encodes the correlations of matching point locations over different face images. The approach is closely related to eigenshapes and linear object classes. In the likelihood part, measurement of point correspondence is based on Gabor filter responses. In addition to matching filter amplitudes, we compute also the discrepancy of the filter phases, which increases the spatial accuracy of the matched locations. The similarity is based on measuring the distribution of Gabor filter responses in a given object location, and finding a low dimensional subspace to model the variation. The probability of match is proportional to the angle between the subspace and the vector of Gabor filter responses. The object matching is carried out as Bayesian inference: the goal is to find the posterior probability distribution of the possible matches given the image and the object prior. We use Markov Chain Monte Carlo (MCMC) methods, such as Gibbs and Metropolis sampling, to estimate the posterior probability of matches. In the paper we present MCMC methods suitable for the matching task and present some preliminary results.

As a laboratory demonstration platform, TUT-I mobile robot provides various experimentation modules to demonstrate the robotics technologies that are involved in remote control, computer programming, teach-and-playback operations. Typically, the teach-and-playback operation has been proved to be an effective solution especially in structured environments. The path generated in the teach mode and path correction in real-time using path error detecting in the playback mode are demonstrated. The vision-based image database is generated as the given path representation in the teaching procedure. The algorithm of an online image positioning is performed for path following. Advanced sensory capability is employed to provide environment perception. A unique omni directional vision (omni-vision) system is used for localization and navigation. The omni directional vision involves an extremely wide-angle lens, which has the feature that a dynamic omni-vision image is processed in real time to respond the widest view during the movement. The beacon guidance is realized by observing locations of points derived from over-head features such as predefined light arrays in a building. The navigation approach is based upon the omni-vision characteristics. A group of ultrasonic sensors is employed for obstacle avoidance.

Intelligent image processing will take a more and more important part of the future opto-electronic sensor business. To filter relevant information out of a large data base and to memorize selectively the most important messages become critical features and the camera-like systems cannot reach the goal. A cortical image processor, similar to how human vision and perception are functioning, is requested. This particularly in a world where global video- communication becomes a ubiquitous standard. The search for new filtering methods and more intelligent visual information processors therefore remains an item of topical interest for R and D activities.

An intelligent robot is a remarkably useful combination of a manipulator, sensors and controls. The use of these machines in factory automation can improve productivity, increase product quality and improve competitiveness. This paper presents a discussion of recent and future technical and economic trends. During the past twenty years the use of industrial robots that are equipped not only with precise motion control systems but also with sensors such as cameras, laser scanners, or tactile sensors that permit adaptation to a changing environment has increased dramatically. Intelligent robot products have been developed in many cases for factory automation and for some hospital and home applications. To reach an even higher degree of applications, the addition of learning may be required. Recently, learning theories such as the adaptive critic have been proposed. In this type of learning, a critic provides a grade to the controller of an action module such as a robot. The adaptive critic is a good model for human learning. In general, the critic may be considered to be the human with the teach pendant, plant manager, line supervisor, quality inspector or the consumer. If the ultimate critic is the consumer, then the quality inspector must model the consumer's decision-making process and use this model in the design and manufacturing operations. Can the adaptive critic be used to advance intelligent robots? Intelligent robots have historically taken decades to be developed and reduced to practice. Methods for speeding this development include technology such as rapid prototyping and product development and government, industry and university cooperation.

A single rotating sonar element is used with a restricted angle of sweep to obtain readings to develop a range map for the unobstructed path of an autonomous guided vehicle (AGV). A Polaroid ultrasound transducer element is mounted on a micromotor with an encoder feedback. The motion of this motor is controlled using a Galil DMC 1000 motion control board. The encoder is interfaced with the DMC 1000 board using an intermediate IMC 1100 break-out board. By adjusting the parameters of the Polaroid element, it is possible to obtain range readings at known angles with respect to the center of the robot. The readings are mapped to obtain a range map of the unobstructed path in front of the robot. The idea can be extended to a 360 degree mapping by changing the assembly level programming on the Galil Motion control board. Such a system would be compact and reliable over a range of environments and AGV applications.

The purpose of this paper is to describe a theory that defines an open method for solving 3D visual data processing and artificial intelligence problems that is independent of hardware or software implementation. The goal of the theory is to generalize and abstract the process of 3D visual processing so that the method can be applied to a wide variety of 3D visual processing problems. Once the theory is described a heuristic derivation is given. Symbolic processing methods can be generalized into an abstract model composed of eight basic components. The symbolic processing model components are: input data; input data interface; symbolic data library; symbolic data environment space; relationship matrix; symbolic logic driver; output data interface and output data. An obstacle detection and avoidance experiment was constructed to demonstrate the symbolic processing method. The results of the robot obstacle avoidance experiment demonstrated that the mobile robot could successfully navigate the obstacle course using symbolic processing methods for the control software. The significance of the symbolic processing approach is that the method arrived at a solution by using a more formal quantifiable process. Some of the practical applications for this theory are: 3D object recognition, obstacle avoidance, and intelligent robot control.

The purpose of this paper is to present a real-time vision system for position determination and vision guidance to navigate an autonomous mobile robot in a known environment. We use a digital camera, which provides ten times the video capture bandwidth than a USB, using FireWire interface. In order to achieve real-time image processing we use MMX technology to accelerate vision tasks. Camera calibration is a necessary step in 3D computer vision in order to extract metric information from 2D images. Calibration is used to determine the camera parameters by minimizing the mean square error between model and calibration points. The camera calibration we use here is based on several views of a planar calibration pattern, which is an easy-to-use and accurate algorithm. For position pose of the robot we use the corner points and lines, features extracted from the image, and matched with the model of the environment. The algorithm is as follows: first we compute an initial pose using the Ganatipathy's four-point algorithm, and we use this initial estimation as the starting point for the iterative algorithm proposed by Araujo in order to refine our pose.

In this paper, we describe the 3D shape reconstructing system from multiple view images using octree and silhouette. Our system consists of four calibrated cameras. Each camera is connected to a PC that locally extracts the silhouettes from the image captured by the camera. The four silhouette images and camera images are then sent to host computer to perform 3D reconstruction. For making the reconstruction faster, the object 3D space is represented by octree structure. If an octant does not entirely consist of the same type of voxels, then it is further subdivided until homogeneous cubes, possibly single voxels, are obtained. Allocating these cubes, and projecting them into all silhouette images, we perform the intersection of the projected cube region with silhouette region. We develop a new algorithm for fast speed constructing octree. The algorithm can reduce time complexity to check if a node should project 8 cube vertices to image plane, using a stack that keeps parents' temporary cube type. By using our algorithm, our system runs in semi real time computation, (about 5 frames per second) for generating 3D shape of the human in voxel representation.

Recently, there is an increasing interest in capturing 3D models of real objects. The range scanner can acquire high quality shape data of the object, but the texture image for surface rendering obtained by the scanner is not generally high resolution and high quality. High-resolution color images at different position are generally taken in addition to the range data so that more realistic images can be rendered from the captured 3D model using such high quality textures. We propose the method of modeling high quality 3D model in shape and appearance by aligning multiple view range images obtained by a range scanner and multiple view color images taken by a digital camera around the object. Color images used as textures are calibrated by Tsai's method, in which lens distortion is also calibrated. On the other hand, a surface model of the object is created by registering and integrating range data sets taken from multiple directions. For registration, we use color ICP (Iterative Closest Point) algorithm that aligns two surfaces using color images and surface shape of the object. For integration, we build voxel model from range images and then detect polygons by Marching Cubes algorithm. We apply textures of high-resolution images to the surfaces by blending, and finally reconstruct realistic 3D model concerned with shape and appearance.

This paper describes a new approach for the motion estimation problem of digital video based upon the lowest requirement for residual encoding. Most Encoders today use Mean Absolute Difference (MAD) or Mean Square Error (MSE) as the test for the best match. These tests do not give any meaningful measure of how much data may be needed to encode the residuals. Our technique is based on using a new criterion to determine the best match block. The new criteria is based on the minimum number of coefficients to represent the residuals. Experimental results show that, using the new proposed technique we achieve a higher compression ratio for the same quality.

We propose a method for three-dimensional (3D) reconstruction of environment from image sequence taken with handy video camera. The convention EPI analysis is a popular method for 3D reconstruction from motion camera image sequence, but the camera needs to be moved linearly with constant velocity. We propose a method for enabling the EPI analysis to be applicable to image sequence taken with a handy video camera of unknown motion. The proposed method removes irregular shaking by hands from input image sequence by using the factorization method and tracking feature points. We apply the convention EPI analysis to the corrected image sequence as if which were taken by camera moved horizontally with constant velocity. The method provides 3D points of the feature points in the scene. In addition, we estimate surface of environmental structure from the 3D points obtained with the method. In the surface estimation, the 3D position of the feature points can be adjusted by matching the texture on the input images. Finally, the 3D model represented by triangle meshes is reconstructed. By rendering the corresponding image region to each mesh, the proposed method made it possible to reconstruct 3D scene with more reality than traditional one.

This project investigates the feasibility of creating real-time 3D images, using data obtained from satellites. The main components that will be discussed are stereoscopic vision, 3D display devices, and satellite imaging. 3D display devices based on the principles of human stereoscopic vision do exist. A model for converting satellite images to suitable input for such a 3D display is proposed. It is possible to obtain stereoscopic image pairs from some satellites. In cases where this is not possible, the model proposes using an original image and a satellite height map (obtained from different sources) to create a stereoscopic image pair.

We present results of using Bayesian inference for recovering the 3D shape and texture of an object based on information extracted from a single 2D image. We are using a number of different models for specific object classes. The goal is to combine the classes to a hierarchical structure. Instead of searching for the most probable explanation we estimate the entire posterior distribution of the model parameters using Markov chain Monte Carlo methods. The evaluation of model fit is based on combining edge information with intensity difference between the model and the target image.

This paper describes a method for automatic registration of real-time multi-view range images obtained under the condition free from any artificial restriction. Firstly, we derive affine transformation matrix, which gives mutual position relationship between two consecutive range images, from automatically extracted four corresponding points between them. Then, concatenating thus derived affine transformation matrices in order, we make registration all range images by transforming them to one coordinate system of arbitrary chosen range image. In extracting four corresponding points automatically, we utilize four cross points between four real and/or virtual planes extracted from extended gaussian-image of the target. As a result, we could reduce error and computational cost and realize automatic registration.

The vast majority of structured light applications are dedicated to range measurement of non-moving objects. In this paper we present a method for obtaining 3D information from a complex scene with moving objects using a simple structured light pattern, namely a dot matrix. The major problem of structured light techniques - the correspondence problem - is usually solved either by involving a sophisticated pattern (color, thickness, code) or by taking advantage of the spatial properties of the pattern and the scene (rotation of the pattern, restricting the working volume, ordering constraint). Our approach enhances the spatial methods by involving the temporal properties of the scene by tracking the projected dot pattern over a sequence of images. Due to this combination of spatial and temporal information, the method solves the correspondence problem very robustly and up to 10 times faster than an algorithm based on spatial information only. Dot splitting, dot disappearance and dot occlusions are detected. We demonstrate our method on several real sequences. In order to evaluate the performance of the method, we performed extensive evaluation on simulated data. The presented method for obtaining coarse temporal 3D information is very simple and fast, which makes it attractive for various surveillance applications.

A new optical arrangement is introduced for such applications of machine vision that need to discriminate structured laser lighting from stray ambient lighting. Instead of using transmissive filters that usually have high attenuation, a double reflective filter arrangement is presented. A combination of dielectric and metallic mirrors achieves steep stopband attenuation especially in the near infrared region where Si detectors are sensitive and common blackbody-like radiators have high spectral density. An additional advantage follows from the compact size of such a Maksutov-type mirror telescope. Compared to a corresponding refractive design, the length is reduced to less than half.

While designing mobile robots, we do think that the prototyping phase is really critical. Good and clever choices have to be made. Indeed, we may not easily upgrade such robots, and most of all, when the robot is on its own, any change in both the software and the physical body is going to be very difficult, if not impossible. Thus, a great effort has to be made when prototyping the robot. Furthermore, I think that the kind of programming is very important. If your programming model is not expressive enough, you may experience a great deal of difficulties to add all the features you want, in order to give your robot reactiveness and decision making autonomy. Moreover, designing, and prototyping the on-board software of a reactive robot brings other difficulties. A reactive robot does not include any matter of rapidity. A reactive system is a system able to respond to a huge panel of situations of which it does not have the schedule. In other words, for instance, the robot does not know when a particular situation may occur, and overall, what it would be doing at this time, and what would be its internal state. This kind of robot must be able to take a decision and to act even if they do not have all the contextual information. To do so, we use a computer language named oRis featuring object and active object oriented programming, but also parallel and dynamic code, (the code can be changed during its own execution). This last point has been made possible because oRis is fully interpreted. However oRis may call fully compiled code, but also Prolog and Java code. An oRis program may be distributed on several computers using TCP/IP network connections. The main issue in this paper is to show how active objet oriented programming, as a modern extension of object oriented programming, may help us in designing autonomous mobile robots. Based on a fully parallel software programming, an active object code allows us to give many features to a robot, and to easily solve tasks conflicts. Active object oriented programming is also very useful in matter of software engineering. Indeed, inside the code, the separation between the logical parts is explicit and plain. So it allows the designer to take only the robot's logical software part, regardless of the software testing environment, and to put it on the physical robot. And even among the logical parts of the robot software, the separation is quite huge, which is a good thing in terms of code engineering, upgrading and reusing. This kind of approach is, or should be, imposed by the particular constraints that lie on military robots, and on any kind of autonomous systems acting in hostile environments, if not in really unknown environments. These systems have to lead a mission on which other systems, and even human lives, rely on. That is the reason why we want to have an accurate look on the on-board software which ensures the robot's autonomy of decision.

Research in this visual servoing field in the past few decades has produced remarkable results, leading to many exciting expectations as well as new challenges. However, because of the complicated calculation of the inverse Jacobian, it is difficult to implement in real time. Therefore, instead of using the inverse Jacobian, this paper employs the ANFIS (Adaptive Neuro Fuzzy Inference System) approach for visual servo control of a robot manipulator. It is based on visual feedback and no prior information about the kinematics of robot and the camera calibration are unnecessary. Firstly, to efficiently control a manipulator, 3D space is divided into two 2D spaces. And then, we acquire training data from each 2D space and ANFIS is learned by the training data. We categorize the robot movement into two kinds of actions. That is, TOWARD action is performed, in the xy plane, by joint 1 and APPROACH action is performed, in the plane orthogonal to the xy plane, by joint 2 and joint 3. The time varying object can be tracked by controlling both actions in each plane and the simulation results show the validation of our approach.

For robust and secure behavior in natural environment an autonomous vehicle needs an elaborate vision sensor as main source of information. The vision sensor must be adaptable to the external situation, the mission, the capabilities of the vehicle and the knowledge about the external world accumulated up to the present time. In the EMS-Vision system, this vision sensor consists of four cameras with different focal lengths mounted on a highly dynamic pan-tilt camera head. Image processing, gaze control and behavior decision interact with each other in a closed loop. The image processing experts specify so-called regions of attention (RoAs) for each object in 3D object coordinates. These RoAs should be visible with a resolution as required by the measurement techniques applied. The behavior decision module specifies the relevance of obstacles like road segments, crossings or landmarks in the situation context. The gaze control unit takes all this information in order to plan, optimize and perform a sequence of smooth pursuits, interrupted by saccades. The sequence with the best information gain is performed. The information gain depends on the relevance of objects or object parts, the duration of smooth pursuit maneuvers, the quality of perception and the number of saccades. The functioning of the EMS-Vision system is demonstrated in a complex and scalable autonomous mission with the UBM test vehicle VAMORS.

The analysis of the prenatal engineering of the human brain and more specifically that of the human eye may encourage new interpretations and better understanding of cortical processors and lead to better ideas about how to build optical sensors. What human vision at its first processing stages realizes is an adaptive transformation of physical parameters from an outer 4D-spatiotemporal into an inner psychological world or its reciprocal projection and construction of an illusionary (inner or outer) world. The description of some of the most remarkable steps in the development of the human eye before birth, very critical for the optical functionalities in vision, will illustrate the new interpretations.

The logical structure of robot vision systems is introduced. It can be used as a basis for designing such systems. Both sensor data fusion and knowledge representation in a vision system may be broken down into four hierarchical levels where at each level knowledge representation in the form of partial models and data fusion complement each other. Multi-processor architectures supporting this type of structure are introduced as a particularly efficient basis for robot vision systems. Practical implementation is given.

Vision is a part of a larger information system that converts visual information into knowledge structures. These structures drive vision process, resolving ambiguity and uncertainty via feedback, and provide image understanding, that is an interpretation of visual information in terms of such knowledge models. A computer vision system based on such principles requires unifying representation of perceptual and conceptual information. Computer simulation models are built on the basis of graphs/networks. The ability of human brain to emulate similar graph/networks models is found. That means a very important shift of paradigm in our knowledge about brain from neural networks to the cortical software. Starting from the primary visual areas, brain analyzes an image as a graph-type spatial structure. Primary areas provide active fusion of image features on a spatial grid-like structure, where nodes are cortical columns. The spatial combination of different neighbor features cannot be described as a statistical/integral characteristic of the analyzed region, but uniquely characterizes such region itself. Spatial logic and topology naturally present in such structures. Mid-level vision processes like clustering, perceptual grouping, multilevel hierarchical compression, separation of figure from ground, etc. are special kinds of graph/network transformations. They convert low-level image structure into the set of more abstract ones, which represent objects and visual scene, making them easy for analysis by higher-level knowledge structures. Higher-level vision phenomena like shape from shading, occlusion, etc. are results of such analysis. Such approach gives opportunity not only to explain frequently unexplainable results of the cognitive science, but also to create intelligent computer vision systems that simulate perceptional processes in both what and where visual pathways. Such systems can open new horizons for robotic and computer vision industries.

In this paper, we present the results of the extension of the mathematical morphology to color images by treating multi-channel data as vectors. The approach presented here uses the HSI and related color spaces (intuitives). A modification of the lexicographical order for vectorial processing is developed. The importance of this new method lies on automatic selection of elements of the HSI and related color spaces to form an ordering structure. The achievement of the algorithm is realized through the introduction of a weight factor to reduce the high preference of the first component of the classic lexicographical order. Experimental results demonstrate the improvement of this new method.

Color image segmentation plays an important role in the computer vision and image processing area. In this paper, we propose a novel color image segmentation algorithm in consideration of human visual sensitivity for color pattern variations by generalizing K-means clustering. Human visual system has different color perception sensitivity according to the spatial color pattern variation. To reflect this effect, we define the CCM (Color Complexity Measure) by calculating the absolute deviation with Gaussian weighting within the local mask and assign weight value to each color vector using the CCM values.

A new method of low contrast multispectral, hyperspectral and multiparameter images segmentation is outlined. The one has significant advantage in sensitivity and space resolving power of segmentation in comparison with known methods such as principal component transformation and fuzzy C-means clustering segmentation ones. New method is based on using of two important stages: 1) application virtual long-wave holographic transformation to each separate image of analyzed multispectral sequence (it is needed for increasing sensitivity of further analysis); 2) to each pixel of analyzed multispectral image is compare a virtual nonrecursive digital filter with complex coefficients. The one is characterized by its amplitude-frequency (AFC) and phase-frequency (PFC) characteristics. Information features used for visualization and segmentation are frequencies corresponded to maximum (resonance point) or minimum (antiresonance point) of AFC and group delay function calculated on base PFC. Information possibilities of new method are demonstrated on examples of multispectral remote sensing, various physical nature geophysical fields fusion and multiparameter MRI brain tumor hidden area influence detection.

Obstacle avoidance is one of the most critical factors in the design of autonomous vehicles such as mobile robots. The purpose of this paper is to compare and contrast three different methods for obstacle detection and avoidance. These include fixed mounting of sonar sensors, a rotating sonar sensor and a laser scanner. The three systems have been installed on the BEARCAT mobile robot. Current work is ongoing and was tested in June 2001 at the International Ground Robotics Competition. This test bed system provides experimental evaluation of the tradeoffs among the systems in terms of resolution, range and computation speed as well as mounting arrangements. The significance of this work is in the increased understanding of obstacle avoidance for robot control and the applications of autonomous guided vehicle technology for industry, defense and medicine.

Image matching is one of the fundamental problems of computer vision. Various approaches exist. They differ essentially by extracted primitives, by the best match search strategy, and by final applications. Feature based dense matching methods use such geometric primitives as raw pixels, edges, interest points, etc. Some of the correlation based matching methods involve a distance calculation. A time consuming operation. Its enhancement adds pixel complex photometric characteristics such as gradient direction, local curvature and luminosity local disparity, what increases the matching time, but they are usually very noisy. The matching method noise dependency and data volume can be reduced when improving the interest point robustness. This paper proposes to add to interest point primitive a set (vector) of simple characteristics (geometric and photometric), which are invariant to geometric plan transforms. A matching method based upon these enriched pixels and accumulation array concept is presented as well. These elements are useful for 3D obstacle detection in the ongoing project intelligent glasses, our final application. The intelligent glasses is a vision system for humanoid robot and for blind/visually impaired persons under joint development by Rouen University and Robotics Laboratory in Paris.

In this paper we present a vision-based technique for detecting targets of the environment which has to be reached by an autonomous mobile robot during its navigational task. The targets the robot has to reach are the doors of our office building. Color and shape information are used as identifying features for detecting principal components of the door. In fact in images the door can appear of different dimensions depending on the attitude of the robot with respect to the door, therefore detection of the door is performed by detecting its most significant components in the image. Positive and negative examples, in form of image patterns, are manually selected from real images for training two neural classifiers in order to recognize the single components. Each classifier has been realized by a feed-forward neural network with one hidden layer and sigmoid activation function. Moreover for selecting negative examples, relevant for the problem at hand, a bootstrap technique has been used during the training process. Finally the detecting system has been applied to several test real images for evaluating its performance.

In this paper we describe a method for automatic determination of sensor pose (position and orientation) related to a 3D landmark or scene model. The method is based on geometrical matching of 2D image structures with projected elements of the associated 3D model. For structural image analysis and scene interpretation, a blackboard-based production system is used resulting in a symbolic description of image data. Knowledge of the approximated sensor pose measured for example by IMU or GPS enables to estimate an expected model projection used for solving the correspondence problem of image structures and model elements. These correspondences are presupposed for pose computation carried out by nonlinear numerical optimization algorithms. We demonstrate the efficiency of the proposed method by navigation update approaching a bridge scenario and flying over urban area, whereas data were taken with airborne infrared sensors in high oblique view. In doing so we simulated image-based navigation for target engagement and midcourse guidance suited for the concepts of future autonomous systems like missiles and drones.

The purpose of this paper is to describe the use of Global Positioning Systems (GPS) as geographic information and navigational system for a ground based mobile robot. Several low cost wireless systems are now available for a variety of innovative automobile applications including location, messaging and tracking and security. Experiments were conducted with a test bed mobile robot, Bearcat II, for point-to-point motion using a Motorola GPS in June 2001. The Motorola M12 Oncore GPS system is connected to the Bearcat II main control computer through a RS232 interface. A mapping program is used to define a desired route. Then GPS information may be displayed for verification. However, the GPS information is also used to update the control points of the mobile robot using a reinforcement learning method. Local position updates are also used when found in the environment. The significance of the method is in extending the use of GPS to local vehicle control that requires more resolution that is available from the raw data using the adaptive control method.

In such a space where human workers and industrial robots work together, it has become necessary to monitor a robot motion for the safety. For such robot surveillance, we propose a robot tracking system from multiple view images. In this system, we treat tracking robot movement problem as an estimation problem of each pose parameter through all frames. This tracking algorithm consists of four stages, image generating stage, estimation stage, parameter searching stage, and prediction stage. At the first stage, robot area of real image is extracted by background subtraction. Here, Yuv color system is used because of reducing the change of lighting condition. By calibrating extrinsic and intrinsic parameters of all cameras with Tsai's method, we can project 3D model of the robot onto each camera. In the next stage, correlation of the input image and projected model image is calculated, which is defined by the area of robots in real and 3D images. At third stage, the pose parameters of the robot are estimated by maximizing the correlation. For computational efficiency, a high dimensional pose parameter space is divided into many low dimensional sub-spaces in accordance with the predicted pose parameters in the previous flame. We apply the proposed system for pose estimation of 5-axis robot manipulator. The estimated pose parameters are successfully matched with the actual pose of the robots.

In this research, the recognition of gesture in 3D space is examined by using serial range images obtained by a real-time 3D measurement system developed in our laboratory. Using this system, it is possible to obtain time sequences of range, intensity and color data for a moving object in real-time without assigning markers to the targets. At first, gestures are tracked in 2D space by calculating 2D flow vectors at each points using an ordinal optical flow estimation method, based on time sequences of the intensity data. Then, location of each point after 2D movement is detected on the x-y plane using thus obtained 2D flow vectors. Depth information of each point after movement is then obtained from the range data and 3D flow vectors are assigned to each point. Time sequences of thus obtained 3D flow vectors allow us to track the 3D movement of the target. So, based on time sequences of 3D flow vectors of the targets, it is possible to classify the movement of the targets using continuous DP matching technique. This tracking of 3D movement using time sequences of 3D flow vectors may be applicable for a robust gesture recognition system.

We are developing a nursing system using robots and cameras. The cameras are mounted on a remote controlled monorail robot which moves inside a room and watches the elderly. It is necessary to pay attention to the elderly at home or nursing homes all time. This requires staffs to pay attention to them at every time. The purpose of our system is to help those staffs. This study intends to improve such situation. A host computer controls a monorail robot to go in front of the elderly using the images taken by cameras on the ceiling. A CCD camera is mounted on the monorail robot to take pictures of their facial expression or movements. The robot sends the images to a host computer that checks them whether something unusual happens or not. We propose a simple calibration method for positioning the monorail robots to track the moves of the elderly for keeping their faces at center of camera view. We built a small experiment system, and evaluated our camera calibration method and image processing algorithm.

A model of an object is an image consisting of features of the object. The input is a gray scale image from which features are computed. In his doctoral thesis J. L. Chen used a model based approach for object recognition. His method is based on Rosin's work for extraction of parts. Both model and scene features are contour based properties. Properties of each part such as area, compactness, convexity, etc., are computed and used to match the scene image to the model. This paper extends the algorithm in several directions. The contours are improved using two passes over the initial input image. The notion of internal part or base of an object is introduced and used to normalize the part areas. Insignificant parts are merged with neighboring parts to provide a better segmentation of the scene. Interpretation trees are used to match scene to object. The algorithm is tested on simple hand drawn images and also images of buildings obtained from architectural databases.

Image segmentation is an important component of image processing which is necessary in the early stages of image analysis. Typical methods of image segmentation are utilizing region information. They use statistics, such as the mean and standard deviation of the pixel intensity within sub-images, with the final segmentation being obtained by a succession of splitting and merging processes of sub-images in order to create regions with quasi- homogeneous properties. In this paper, we propose a co- occurrence matrix based method of image segmentation in region-based techniques. It utilizes the observation that features of the multiple windows neighboring a pixel do not differ significantly from one another, and that features corresponding to pixels belonging to the same object form a cluster in the feature space, which may frequently be approximated by a Gaussian distribution. This paper extends the co-occurrence matrix based method. The definition of co- occurrence features is extended from one dimension to many dimensions: the number of observation windows is extended from two to an arbitrary number.

This paper presents a control mechanism for video compression and transmission that relies on spatio-temporal image segmentation and non-uniform resolution representation. In the proposed scheme, a linked pyramid structure is built over each frame. Therefore, the video sequence is processed as a 3D data batch instead of a series of 2D images. The spatio-temporal segmentation is based on the adaptive link principle. The set of links defined between nodes of a single pyramid structure is extended to consecutive pyramids. According to the adaptive link principle, links between consecutive pyramids are rearranged so that a given node at a pyramid is linked to an homogeneous region of pixels at its base, but also to an homogeneous region of pixels at the base of the previous pyramid. Hence, resulting regions are coherent in time but also in space and any region may be extracted and tracked through the sequence by following the link structure. Then, if we assume that the most important regions of the frames are typically those changing more often, they can be represented at high resolution, while the least important regions can be represented at low resolution. This operation allows a severe data volume reduction when transmitting the video signal. Besides, regions presenting few changes from frame to frame do not need to be transmitted as often as moving areas because they will be very similar during a few frames.

The process of automatically finding objects is very important for tracking and grasping objects using computer vision. Object finding is on the one hand a very time consuming process and on the other hand very sensitive to varying environmental conditions. Hence initialization is often performed manually. This work presents a method that operates automatically, threshold-free, and uses only model based information. Region based attributes such as color or texture are used in combination with a multi-spectral classification method to build a gray scale image, where regions with a high likelihood for the object searched are bright and the other parts are dark. To adapt to the ambient lighting situation a color measurement and a full matrix color transform are proposed and evaluated. First test runs show that this approach is fast and robust and enables the successful and automatic object finding.

Intelligent industrial and mobile robots may be considered proven technology in structured environments. Teach programming and supervised learning methods permit solutions to a variety of applications. However, we believe that to extend the operation of these machines to more unstructured environments requires a new learning method. Both unsupervised learning and reinforcement learning are potential candidates for these new tasks. The adaptive critic method has been shown to provide useful approximations or even optimal control policies to non-linear systems. The purpose of this paper is to explore the use of new learning methods that goes beyond the adaptive critic method for unstructured environments. The adaptive critic is a form of reinforcement learning. A critic element provides only high level grading corrections to a cognition module that controls the action module. In the proposed system the critic's grades are modeled and forecasted, so that an anticipated set of sub-grades are available to the cognition model. The forecasting grades are interpolated and are available on the time scale needed by the action model. The success of the system is highly dependent on the accuracy of the forecasted grades and adaptability of the action module. Examples from the guidance of a mobile robot are provided to illustrate the method for simple line following and for the more complex navigation and control in an unstructured environment. The theory presented that is beyond the adaptive critic may be called creative theory. Creative theory is a form of learning that models the highest level of human learning - imagination. The application of the creative theory appears to not only be to mobile robots but also to many other forms of human endeavor such as educational learning and business forecasting. Reinforcement learning such as the adaptive critic may be applied to known problems to aid in the discovery of their solutions. The significance of creative theory is that it permits the discovery of the unknown problems, ones that are not yet recognized but may be critical to survival or success.

There is a gap in most of pattern recognition approaches between the low-level processing and the high-level analysis. A recent approach fills this gap for the problem of structural identification. It supports execution of the low-level image processing routines under full control of an identification task. This paper extends the approach to cover as possible omissions of some object details during detection, as well as partial occlusions.

2D Vision in a 3D System will be an overview of important software and mechanical issues related to designing and implementing a system with 3-dimensional characteristics constrained by a 2-dimensional vision system. A general methodology for choosing the right components, including cameras, optics, and lighting will be discussed. Some choices for image processing techniques, related in particular to calibration and pattern-matching approaches, will be overviewed. The discussion will involve approaches to auto-focusing and additional mathematical algorithms that may likely be required. These issues will include world-to-camera transformations and considerations to take when dealing with rotational attributes. The presentation is intended primarily for software engineers and/or other engineers that would be involved in the prototyping, design, and implementation phases of such a project. Typical examples of such systems will be cited and a more in-depth overview of a particular example will be made. Some typical pitfalls, both mechanical and software-related, will also be broached.

In this paper, we present the implementation of the robust detail preserving filters with impulsive noise removal and multiplicative noise suppression for computer vision applications. The designed filter is the consequential connection of two filters. The first filter uses the value of central pixel of the filtering window to provide the preservation of fine details and the redescending M-estimators combined with the Wilcoxon estimator to provide impulsive noise rejection. The second filter uses the output of the first filter as the pre-estimator for an adaptive calculation in the redescending M-estimator. We investigated various types of restriction functions in the M-estimator those are similar to the ones used in the Sigma filter to provide multiplicative noise suppression. The optimal values of the parameters of designed filters in presence of different noise mixture are determined. We also present the implementation of the proposed filtering algorithms by means of use of DSP TMS320C6701 processor for real time filtering. Different simulation data are presented in the paper and shown the statistical efficiency of the filters and the values of time in case of real time filtering.

In this paper, we try to extend Fisher linear discriminant analysis (FLD) to the singular cases. Firstly, PCA is used to reduce the dimension of feature space to N-1 (N denotes the number of training samples). Then, the transformed space is divided into two subspaces: the null space of within- class scatter matrix and its orthogonal complement, from which two cases of optimal discriminant vectors are selected respectively. Finally, we test our method on ORL face database, and achieve a recognition rate of 97% with a minimum distance classifier or a nearest neighbor classifier. The experimental results indicate that our approach is better than classical Eigenfaces and Fisherfaces with respect to recognition performance.

In this paper, a speech recognition system is developed using higher order statistic (HOS) with its fourth order of crosscorrelation (trispectrum) analysis. To analyze the distribution of the trispectrum data along its two- dimensional representation, we developed an adaptive feature extraction mechanism of the trispectrum speech data based on cascade neural network that consists of SOFM (Self-Organizing Feature Map) and LVQ (Learning Vector Quantization). This cascade neural network is used as an adaptive codebook generation algorithm for determining the feature distribution of the trispectrum speech data. Two types of neural networks, namely back-propagation neural network and probabilistic neural networks, are then used as the pattern classifier of this speech recognition system. Comparison of the recognition system using those neural networks as the classifier is conducted based on sample data with and without Gaussian noise. Experimental result showed that PNN has superior recognition rate compared with that of BPNN, especially when a harsh condition of noise is added to the system.

We have proposed a boundary extraction algorithm for cross-sectional critical dimension (CD) measurements of the scanning electron microscope (SEM) images in machine/computer vision. Noisy SEM images have the image discontinuity of roof edge with peak. A task is to extract a low contrast boundary or edge with about one pixel width in a noisy cross sectional SEM image. Also CD measurements require that the algorithm locate a boundary or an edge at the correct position. Our policy is to detect the edges with high degree of confidence at first, and then to extract the edges with the high probabilities of edge that are calculated by using gray levels of neighboring pixels. Our proposed algorithm integrates the Gaussian pyramid to suppress noise, morphological operations to detect the edges with high degree of confidence, the relaxation method to recover the gaps in the detected edges, and the hierarchical approach. Application of the proposed algorithm to cross sectional SEM images of DRAMs shows its validity.

Typical surface inspection tasks using RGB vision require the analysis of tens of megabytes of image data per second, with low false alarm and error escape rates. Although automatic inspection systems have become more common on production lines, for example in sawmills, there are substantial needs to improve their performance and accuracy. Detection is one very important part of image flow before defect recognition. Detection is used in order to find suspicious regions of the image, containing possibly defective areas, since defect detection has to cope with very high data rates. It has to be based on relative simple methods. In this paper we describe the effect of different thresholding methods in RGB defect detection. Threshold values were calculated for R, G and B channels, difference channels R-G, R-B and G- B and for mean values from R, G and B channels. The analysis was performed for pinewood. Error escape rate and false alarm rates were used as evaluation criteria. In this paper, R and G channel thresholding methods were the best ones.

In this study we use genetic algorithms to generate test surfaces for a proposed structured light 3D vision system in order to estimate the worst case behavior of error tolerances. The object software evaluates surface profiles for measuring volumes of small objects attached on surfaces that are highly constrained while somewhat arbitrarily shaped. The test system tries to find, by using genetic algorithm search, the shape that results the highest relative error of volume. The parameters of the object system to be optimized include laser angle, image size, object step size, and the number of scan directions. The preliminary results seem to indicate that a genetic algorithm based approach is a beneficial aid in optical system design.

This paper presents methods to improve flexibility and accuracy of deburring of castings. We apply several methods including off-line programming of the deburring paths, accurate localization of the work object, surface measuring of the work object and a force-guided motion control during the deburring task. The eye-in-hand calibration as well as the localization of the work object we carry out using the Bayesian-form estimation method with recursive sensor fusion. As a result from the work object localization we obtain a 3 DOF location increment (position difference between the simulation model and actual workcell) and actual deburring paths are corrected using that increment. The simulation phase includes octree-based collision check and the faceting uses octree principle too. The paper includes results from actual tests which are promising. The methods are designed to be easy-to-implement in any industrial robot.

In industrial inspection one of the key areas is detection of defects from textured surfaces. The goal is to differentiate between a good, normal surface texture and a defected surface texture. In this paper this is achieved with a two-class classifier that is taught only with fault-free samples of surface texture. An unsupervised segmentation scheme is formulated where an unknown sample is classified as a defect if it differs enough from the estimated distribution of texture features extracted from fault-free samples. The extension of the self-organizing map (SOM) algorithm, the so-called statistical SOM, is used to estimate the distribution. Different versions of the statistical SOM are introduced and their computational requirements are discussed. The proposed methods are shown to perform well in segmentation of texture surface images with different kinds of defects.

In the last years the detection and classification of surface defects of material is assuming great importance. Visual inspection can help to increase the product quality and, in particular context, the maintenance of products. The railway infrastructure is a particular field in which the periodical surface inspection of rolling plane can help an operator to prevent critical situation. We use a Gabor filter to emphasize the image regions with grey level variation. The Gabor filter h(x,y) is characterized by a frequency F, direction (theta) and parameter (sigma) . We have selected experimentally four filters with directions 0, (pi) /4, (pi) /2 and (pi) 3/4 with F equals (root)2/8 cycle/pixel and (sigma) equals 2. The problem of detection and classification is a crucial part of our work because cannot be defined an exhaustive training set of defect and no-defect images. It is necessary a method able to self-learn changes. Investigating about this problem we propose in the paper a novel Self Organized Map (SOM) network, appropriately modified, for detection and classification of rail defects. The proposed SOM network learns to classify input vectors according to how they are grouped in the input space. So, SOM learns both the distribution and topology of the input vectors belonging to the training set. During the training phase, the neurons in the layer of an SOM form some cluster or bubble representing the input training with minimum distance among them. The novelty is to modify the SOM network in order to learn continuously during the test phase.

We propose context-aware mobile systems for managing and using services on the behalf of the user. Context-aware mobile systems perceive environmental signals, infer the context (i.e. the state of the system and its local environment) from these signals, and calculate appropriate actions for the detected context. In its simplest form, such a system can be a mobile telephone adjusting its profile based on the noise level and brightness of the environment. A personal robot equipped with a vision system and a manipulator is a more complex example of a context-aware mobile system. Context-aware mobile systems both manage the available services and use them on behalf of the user or guide the user in using them. A portable system offers a single interface for a variety of services. The role of a personal robot is to enhance some services so as to make them more suitable to the user. Both portable and robotic systems are controlled by a system capable of recognizing the user's context and reasoning out actions that would optimally serve the user in the situation at hand. In this paper, we will present our software architecture for context-aware service management and utilization, our approach for controlling personal robots, and our work on virtual and natural interfaces for interacting with a personal robot.

Correlation methods represent a well-known and reliable approach to detect similarities between images, signal patterns, and stochastic processes. However, by means of the cross-correlation function only linear similarities are registered. Unfortunately, often it is not possible to avoid non-linearities in the characteristics of the sensors used or, as in image processing, in the interaction between illumination and the scene to be captured. Thus, in such cases correlation methods may yield poor results. In this paper, we describe alternative strategies to enhance the performance of correlation methods even when the statistical connection between the signals is non-linear. To reduce the impact of non-linearities on the signals to be analyzed, a preprocessing is performed in which certain properties affecting first-order statistics are manipulated. This step impresses the same histogram to the signals to be compared, so that typically higher correlation coefficients are obtained as compared to if no preprocessing methods were used. The performance of our approach is demonstrated with two different tasks. First, a preprocessing strategy is proposed for signals obtained from train-based sensors to enable an identification of rail switches. Finally, a method for comparing striation patterns in forensic science is presented. To investigate the benefit of this approach, a large database of toolmarks is used.

This paper presents incorporation of certain human vision properties in the image processing methodologies, applied in the vision substitutive system for human blind. The prototype of the system has digital video camera fixed in a headgear, stereo earphone and a laptop computer, interconnected. The processing of the captured image is designed as human vision. It involves lateral inhibition, which is developed using Feed Forward Neural Network (FFNN) and domination of the object properties with suppression of background by means of Fuzzy based Image Processing System (FLIPS). The processed image is mapped to stereo acoustic signals to the earphone. The sound is generated using non-linear frequency incremental sine wave. The sequence of the scanning to construct the acoustic signal is designed to produce stereo signals, which aids to locate the object in horizontal axis. Frequency variation implies the location of object in the vertical axis. The system is tested with blind volunteer and his suggestion in formatting, pleasantness and discrimination of sound pattern were also considered.

In a multi-agent system, the action selection strategy is important for the cooperation and coordination of multi agents. However the overlap of actions selected individually by each robot makes the acquisition of cooperation behaviors less efficient. In addition to that, a complex and dynamic environment makes cooperation even more difficult. So in this paper, we propose a control algorithm which enables each robot to determine the action for the effective cooperation in multi-robot system. We employ a reinforcement learning in order to choose a proper action for each robot in its action subspace. In this paper, robot soccer system is adopted for the multi-robot environment. To play a soccer game, elementary actions such as shooting and passing must be provided. Q-learning, which is one of the popular methods for reinforcement learning, is used to determine what actions to take. Through simulations, the efficiency of own proposed algorithm is verified for the cooperation in multi robot system.

In this paper, we introduce a geometric registration method for augmented reality (AR) and an application system, interior simulator, in which a virtual (CG) object can be overlaid into a real world space. Interior simulator is developed as an example of an AR application of the proposed method. Using interior simulator, users can visually simulate the location of virtual furniture and articles in the living room so that they can easily design the living room interior without placing real furniture and articles, by viewing from many different locations and orientations in real-time. In our system, two base images of a real world space are captured from two different views for defining a projective coordinate of object 3D space. Then each projective view of a virtual object in the base images are registered interactively. After such coordinate determination, an image sequence of a real world space is captured by hand-held camera with tracking non-metric measured feature points for overlaying a virtual object. Virtual objects can be overlaid onto the image sequence by taking each relationship between the images. With the proposed system, 3D position tracking device, such as magnetic trackers, are not required for the overlay of virtual objects. Experimental results demonstrate that 3D virtual furniture can be overlaid into an image sequence of the scene of a living room nearly at video rate (20 frames per second).

The object of the paper is to describe how a linear Fresnel lens is designed using I-DEAS (Integrated design, engineering and analysis software) package. The design is carried out by drawing three curves from the focal point at specific radii. A vertical line is drawn at the extreme right end of the curve. By drawing a tangent and a normal at that point, the refracted angle is found out using Snell's law. The refracted ray is extended to meet the arc. Then suitable angles of incidence and refraction are found at that point such that Snell's law is satisfied. The new normal is drawn and the perpendicular to it forms a lens tooth. Equidistant points are plotted on the lens profile and by Lagrange's interpolation the groove angles at each point are determined. By drawing the lens tooth at the suitable groove angles, one half of the lens profile is created which is then mirrored and extruded to get the full profile. The significance of this work is that it ensures the precise focusing of the entire incident light along a straight line without any scatter. This reduces the effective photovoltaic cell area needed and hence cuts the cell cost tremendously. The thickness of the lens is 4 mm, which enables it to be bent and mounted easily.

In this work, a camera with integrated image processing capabilities is discussed. The camera is based on an imager coupled to an FPGA device (Field Programmable Gate Array) which contains an architecture for real-time computer vision low-level processing. The architecture can be reprogrammed remotely for application specific purposes. The system is intended for rapid modification and adaptation for inspection and recognition applications, with the flexibility of hardware and software reprogrammability. FPGA reconfiguration allows the same ease of upgrade in hardware as a software upgrade process. The camera is composed of a digital imager coupled to an FPGA device, two memory banks, and a microcontroller. The microcontroller is used for communication tasks and FPGA programming. The system implements a software architecture to handle multiple FPGA architectures in the device, and the possibility to download a software/hardware object from the host computer into its internal context memory. System advantages are: small size, low power consumption, and a library of hardware/software functionalities that can be exchanged during run time. The system has been validated with an edge detection and a motion processing architecture, which will be presented in the paper. Applications targeted are in robotics, mobile robotics, and vision based quality control.

Convolution of an image is indispensable in many image processing applications, but it is a time-consuming process. In general the convolution mask is restricted the size of 15 by 15 because of its computation time. Many approaches are attempted to reduce the convolution processing time using hardware and software algorithms. But they are restricted in specific application. In this paper, a novel approach is presented. This method is realized by simplifying the convolution process. The convolution mask is approximated and decomposed to more simple form, K convolutions with constant mask value respectively. K is the number of levels of approximation, which is less than or equal to (N+1)/2, where N is the original mask size. Calculation process is reconstructed to reduce recursive multiplications. And cumulative image, which contains sum of pixel values of the rectangle area from the origin to each pixel, is prepared for this process. Processing time is dramatically reduced and resulting image is similar to the one by original convolution mask. For 13 by 13 mask convolution, new method is above 20 times faster than conventional one.