A combined achromatic and chromatic contrast metric for digital images and video is presented in this paper. Our work is aimed at tuning any parametric rendering algorithm in an automated way by computing how much details an observer perceives in a rendered scene. The contrast metric is based on contrast analysis in spatial domain of image sub-bands constructed by pyramidal decomposition of the image. The proposed contrast metric is the sum of the perceptual contrast of every pixel in the image at different detail levels corresponding to different viewing distances. The novel metric shows high correlation with subjective experiments. Important applications involve optimal parameter set of any image rendering and contrast enhancement technique or auto exposure of an image capturing device.

Image fidelity refers to the ability of discriminating the differences between two images, while image quality refers to the preference of one image over another image. Many image fidelity predictors adapted the results from contrast discrimination experiments. It is questionable whether these models can be used for image quality assessment. Image quality judgment mostly occurs to suprathreshold images and the judgment generally cares about the image appearance rather than the subtle differences between two images. In this report, we presented psychophysical evidence that image perception and discrimination have different mechanism in terms of spatial interactions. We built an image processing model of the primary visual system to simulate contrast perception and contrast discrimination at the suprathreshold level. By introducing different neuronal integration mechanisms (i.e., decision rules) to the same model structure, the model was able to predict the perceived contrast and contrast discrimination thresholds of a target embedded in a complex image. Our simulations showed that the contextual effect was significant in contrast perception and thus should not be ignored by the models of image quality measurement. With further calibrations, the 2-D output of the model can be used to as an image quality metric to predict perceptual differences in images.

One goal of image quality modeling is to predict human judgments of quality between image pairs, without needing knowledge of the image origins. This concept can be thought of as device-independent image quality modeling. The first step towards this goal is the creation of a model capable of predicting perceived magnitude differences between image pairs. A modular color image difference framework has recently been introduced with this goal in mind. This framework extends traditional CIE color difference formulae to include modules of spatial vision and adaptation, sharpness detection, contrast detection, and spatial localization. The output of the image difference framework is an error map, which corresponds to spatially localized color differences. This paper reviews the modular framework, and introduces several new techniques for reducing the multi-dimensional error map into a single metric. In addition to predicting overall image differences, the strength of the modular framework is its ability to predict the distinct mechanisms that cause the differences. These mechanisms can be thought of as attributes of image appearance. We examine the individual mechanisms of image appearance, such as local contrast, and compare them with overall perceived differences. Through this process, it is possible to determine the perceptual weights of multi-dimensional image differences. This represents the first stage in the development of an image appearance model designed for image difference and image quality modeling.

The objective of this investigation is to establish whether it is possible to produce a practical forensic tool that can identify the production sources of printed text from different digital print engines. The identification of print using an automated machine system is important because although expert observers can be employed for this task, there are cases when they make mistakes or do not possess the required knowledge. Therefore the development of an automated print identification system is under consideration. It is envisaged that the system will be useful in solving criminal cases involving the analysis of fraudulent replication of official documents, threatening letters and the counterfeiting of consumer products. The methodology used in this investigation employed a digital image analysis system and specially developed software to measure the shape characteristics of text characters. The information about the shapes of the text characters can be stored in a database along with the corresponding data about the print engines that produced them. A database search engine can then be used to classify text characters of unknown origin. The paper will report on the methodology and techniques used in the investigation and the latest experimental results for the project.

Visually lossless image compression algorithms aim to keep the compression-induced distortions below the threshold of visual detection, most-often by exploiting the fact that contrast sensitivity varies with spatial frequency. However, when an image is coded in a visually lossy manner, there is little evidence to suggest that visual quality is preserved by minimizing the compression-induced distortions. This paper presents a visually lossy wavelet image compression algorithm based on contrast allocations and visual global precedence: subbands are quantized such that the distortions in the reconstructed image exhibit specific root-mean squared contrast ratios, and such that edge structure is preserved across scale-space, with a preference for global spatial scales. A model which relates contrast (of the distortions) in the reconstructed image to mean-squared error in the wavelet subbands is derived and presented; this model provides an efficient means of adjusting contrast in the transform domain via traditional quantization techniques, thus allowing the algorithm to be used in a wide variety of coders.

We want to integrate colourfulness in an image quality evaluation framework. This quality framework is meant to evaluate the perceptual impact of a compression algorithm or an error prone communication channel on the quality of an image. The image might go through various enhancement or compression algorithms, resulting in a different -- but not necessarily worse -- image. In other words, we will measure quality but not fidelity to the original picture. While modern colour appearance models are able to predict the perception of colourfulness of simple patches on uniform backgrounds, there is no agreement on how to measure the overall colourfulness of a picture of a natural scene. We try to quantify the colourfulness in natural images to perceptually qualify the effect that processing or coding has on colour. We set up a psychophysical category scaling experiment, and ask people to rate images using 7 categories of colourfulness. We then fit a metric to the results, and obtain a correlation of over 90% with the experimental data. The metric is meant to be used real time on video streams. We ignored any issues related to hue in this paper.

As color is more widely used to carry visual information in the multimedia content, ability to perceive color plays a crucial role in getting visual information. Regardless of color vision variations, one should have visual information equally. This paper proposes the adaptation technique for color vision variations in the MPEG-21 Digital Item Adaptation (DIA). DIA is performed respectively for severe color vision deficiency (dichromats) and for mild color vision deficiency (anomalous trichromats), according to the description of user characteristics about color vision variations. Adapted images are tested by simulation program for color vision variations so as to recognize the appearance of the adapted images in the color deficient vision. Experimental result shows that proposed adaptation technique works well in the MPEG-21 framework.

We carried out a number of subjective experiments using typical streaming content, codecs, bitrates and network conditions. In an attempt to review subjective testing procedures for video streaming applications, we used both Single Stimulus Continuous Quality Evaluation (SSCQE) and Double Stimulus Impairment Scale (DSIS) methods on the same test material. We thus compare these testing methods and present an analysis of the experimental results in view of codec performance. Finally, we use the subjective data to corroborate the prediction accuracy of a real-time non-reference quality metric.

Many current quality evaluation models were designed to produce a single estimate of perceived quality for a video sequence coded at relatively high rates. These metrics perform a multi-channel decomposition to simulate the processes of the Human Visual System (HVS), followed by a distortion pooling stage that collapses the channels over frequency, time and space. Estimating quality at short intervals over the length of a video sequence, however, may be more useful for long video sequences than a single estimate, particularly in such applications as two pass video coding and video quality monitoring. This paper presents an objective metric designed to perform this task on video sequences coded at low bit rates. The metric implements a wavelet transform-based model of the human visual system and a method of temporal error pooling suited to continuous estimation of perceived quality. A time series distance metric based on piecewise linear representations is also introduced in order to quantify performance. The metric is evaluated on a wide range of low bit rate video content and shown to perform well in terms of the shape and overall mean of the output perceived quality waveform.

This work addresses the problem of studying and characterizing individual artifacts found in digital video applications (e.g., blockiness, blurriness). In particular, the goal of this paper was to examine the properties of synthetic blocky and blurry artifacts, designed to be relatively pure, and a combination of these two artifacts. We performed a psychophysical experiment in which human subjects were asked to detect these artifacts, identify their types, and rate their annoyance. In most cases, the blocky and blurry artifacts were identified as blocky and blurry, respectively. In combined blocky-blurry artifacts the salience of blockiness increased and blurriness decreased as artifact strength increased. The blocky artifacts produced higher annoyance values than the blurry ones when the total squared error was the same.

The objective of pre-processing is to eliminate information of least visual significance prior to encoding, in order to achieve best overall performance of a video compression system. We formulate the pre-processing problem in an operational rate-distortion framework, with the aim of maximizing the visual quality of the compressed video, as well as the coding efficiency of the system. Rather than filtering the original video, our novel approach consists of filtering the motion compensated error signal. This offers a significant computational advantage over other pre-filtering methods without sacrificing effectiveness. The proposed method selects the parameters of a pre-filter in conjunction with the selection of the quantization scales of the encoder. We incorporate a perceptual quality metric in the optimization process, in order to maximize the visual quality of the compressed video for given rate constraints. Our approach is developed for motion-compensated block-based discrete cosine transform coders, which form the basis of several video coding standards. We use the visual quality metric that was proposed by Watson, which was developed for such coders, based on the discrete cosine transform decomposition. The effectiveness of the proposed approach is demonstrated by simulation results, carried out within the framework of MPEG-2 video compression.

Traditional color appearance modeling has recently matured to the point that available, internationally-recommended models such as CIECAM02 are capable of making a wide range of predictions to within the observer variability in color matching and color scaling of stimuli in somewhat simplified viewing conditions. It is proposed that the next significant advances in the field of color appearance modeling will not come from evolutionary revisions of these models. Instead, a more revolutionary approach will be required to make appearance predictions for more complex stimuli in a wider array of viewing conditions. Such an approach can be considered image appearance modeling since it extends the concepts of color appearance modeling to stimuli and viewing environments that are spatially and temporally at the level of complexity of real natural and man-made scenes. This paper reviews the concepts of image appearance modeling, presents iCAM as one example of such a model, and provides a number of examples of the use of iCAM in still and moving image reproduction.

Experiments were conducted to investigate colour appearance under mesopic vision. Lightness, colourfulness and hue observations of 40 test colours were accumulated for eight phases with four different luminance levels covering 0.1 to 90 cd/m² and two different stimulus sizes corresponding to viewing angles of 2° and 10° using the magnitude estimation method. The psychophysical effects of luminance level and patch size on colour appearance were investigated and the role of the rods under mesopic vision was explored.

This paper reports further progress on a computational model of human achromatic color perception first presented at Human Vision and Electronic Imaging VI. The model predicts the achromatic colors of regions within 2D images comprising arbitrary geometric arrangements of luminance patches separated by sharp borders (i.e., Land Mondrian patterns). The achromatic colors of regions of homogeneous luminance are computed from the log luminance ratios at borders. Separate lightness and darkness induction signals are generated at the locations of borders in a cortical representation of the image and spread directionally over several degrees of visual angle. The color assigned to each point in the image is a weighted sum of all of the lightness and darkness signals converging on that point. The spatial convergence of induction signals can be modeled as a diffusive color filling-in process and realized in a neural network. The model has previously been used to predict lightness matches in psychophysical experiments conducted with stimuli consisting of disks and surrounding rings. Here a formal connection is made between the model equations used to predict lightness matches in these experiments and Stevens' power law model of the relationship between brightness and physical intensity. A neural mechanism involving lateral interactions between neurons that detect borders and generate spreading achromatic color induction signals proposed to account for observed changes in the parameters of the model, including the brightness law exponent, with changes in surround size.

Chromatic adaptation transforms generally rely on a variant of the von Kries transformation-method to account for changes in the LMS cone signals that occur when changing from one illuminant to another. Von Kries adaptation also often referred to as the coefficient rule method or the diagonal transformation method-adjusts the 3 color channels by independent scale factors. Since there generally are only 3 known quantities available, namely the ratio of the cone signals of the two adapting illuminants, a crucial aspect of the von Kries method is that it requires only 3 parameters to be specified. A 9-parameter, 3x3 matrix transformation would be more accurate, but it is generally not possible to determine the extra parameters. This paper presents a novel method of predicting the effect a change of illumination has on the cone signals, while still relying on only 3 parameters. To begin, we create a large set of 3x3 matrices representing illuminant changes based on a sizable database of typical illuminant spectra and surface spectral reflectances. Representing these 3x3 matrices as points in a 9-dimensional space, we then apply principal components analysis to find a 3-dimensional basis which best approximates the original matrix space. To model an illumination change, a 3x3 matrix is constructed using a weighted combination of the 3 basis matrices. The relative weights can be calculated based on the 3 standard cone ratios obtained from the illuminant pair. Tests show that the new method yields better results than von Kries adaptation with or without sensor sharpening.

Aerospace information displays require some form of quality assurance due to public safety concerns. However, the color design methodologies, standards, and guidelines employed in development of earlier displays are insufficient to guide current designs, which have reached map-like complexity. For these displays designers can only assure legibility and good attention management with designs that consider relations among all of the graphic elements. Quality assurance needs to focus on performance in context rather than graphic elements. We are investigating web-based color-guidance tools to help advance the design and review processes. Individual performance issues that guidance statements address are demonstrated in the context of a hyperlinked network of supporting infornation at varying levels of technical detail. Draft color designs can be developed and modified in this environment using web-based design tools that visualize the consequences of particular choices and the interactions among the chosen colors.

A complete colorimetric characterization of a cathode ray tube (CRT) display was generated using an all-visual psychophysical measurement procedure. This process requires no external props or matching stimuli and uses a three-stage approach for estimating the display’s (1) system nonlinearities (i.e., gamma), (2) channel luminance ratios (i.e., white-point setting), and (3) phosphor chromaticity matrix. Simple homochromatic brightness matching experiments are used to solve the system nonlinearities using either spatially or temporally modulated stimuli. A novel heterochromatic minimum-flicker technique is presented for estimating the relative luminance contributions of each display channel. Finally, a chromaticity selection process is used to choose an appropriate chromaticity set for the display. These components are used to populate a model of the colorimetric mixing characteristics of the display device. The visually populated display model was compared to an instrumentation-based process. The results of this analysis showed that the all-visual characterization process compared favorably to the instrumentation-based model. For 12 subjects, the mean colorimetric errors (averaged across subjects) for a typical color-management task were approximately 2 CIE ΔE94 units with maximum colorimetric errors (averaged across subjects) of approximately 4 CIE ΔE94 units.

Color is the most frequently used feature in image analysis and retrieval. Over the years color features have been significantly improved, advancing from simple histogram-based descriptors to address more complex features such as dominant colors, important regions and spatial relationships. However, no matter how sophisticated, most of the proposed representations are still unable to cross the well-known semantic gap and when applied to diverse image collections seldom produce satisfactory results. This work analyzes the shortcomings of “traditional” color retrieval approach, and provides a new framework for “capturing” the elementary aspects of image semantics. Our approach is based on the observation that the words we use to describe images and their content represent a true link between the pictorial (low-level) representation and higher-level semantics derived by observers. Following previous experimental findings on the role of color in the perception of basic semantic templates (people, nature, indoor/outdoor, etc.), we propose a new feature representation based on color names. We also address several important steps needed to construct such representation: design of a color-name vocabulary and syntax, design of computational model and color naming metric, and the application to complex images. We also include some results that provide a good match to human judgments, thus validating the use color names to develop semantic descriptors.

Eye movement behavior was investigated for image-quality and chromatic adaptation tasks. The first experiment examined the differences between paired comparison, rank order, and graphical rating tasks, and the second experiment examined the strategies adopted when subjects were asked to select or adjust achromatic regions in images. Results indicate that subjects spent about 4 seconds looking at images in the rank order task, 1.8 seconds per image in the paired comparison task, and 3.5 seconds per image in the graphical rating task. Fixation density maps from the three tasks correlated highly in four of the five images. Eye movements gravitated toward faces and semantic features, and introspective report was not always consistent with fixation density peaks. In adjusting a gray square in an image to appear achromatic, observers spent 95% of their time looking only at the patch. When subjects looked around (less than 5% of the time), they did so early. Foveations were directed to semantic features, not achromatic regions, indicating that people do not seek out near-neutral regions to verify that their patch appears achromatic relative to the scene. Observers also do not scan the image in order to adapt to the average chromaticity of the image. In selecting the most achromatic region in an image, viewers spent 60% of the time scanning the scene. Unlike the achromatic adjustment task, foveations were directed to near-neutral regions, showing behavior similar to a visual search task.

A major challenge in Virtual Reality is to achieve realism at interactive rates. However, the computational time required for realistic image synthesis is significant, precluding such realism in real-time. This paper demonstrates a concept that may be exploited to reduce rendering times substantially without compromising perceived visual quality in interactive tasks. We demonstrate the principle of Inattentional Blindness; when attention is focused on a specific task, items in the scene that are unrelated to the performance of the task literally go unnoticed. Our experiment utilises this principle and varies the rendering quality over the image according to the task at hand. Our results show that observers do not perceive the difference in image quality on objects unrelated to their task. We believe this is due to Inattentional Blindness, as their attention was focused on a task rather than image quality. The difference in rendering was clearly visible when the subjects were asked to pay full attention to spotting the quality differences only. Our results show that Inattentional Blindness may be exploited to reduce rendering times substantially without compromising perceived visual quality in interactive tasks.

Eye movements are an external manifestation of selective attention and can play an important role in indicating which attributes of a scene carry the most pertinent information. Models that predict gaze distribution often define a local conspicuity value that relies on low-level image features to indicate the perceived salience of an image region. While such bottom-up models have some success in predicting fixation densities in simple 2D images, success with natural scenes requires an understanding of the goals of the observer, including the perceived usefulness of an object in the context of an explicit or implicit task. In the present study, observers viewed natural images while their eye movements were recorded. Eye movement patterns revealed that subjects preferentially fixated objects relevant for potential actions implied by the gist of the scene, rather than selecting targets based purely on image features. A proto-object map is constructed that is based on highly textured regions of the image that predict the location of potential objects. This map is used as a mask to inhibit the unimportant low-level features and enhance the important features to constrain the regions of potential interest. The resulting importance map correlates well to subject fixations on natural-task images.

We present a model that predicts saccadic eye-movements and can be tuned to a particular human observer who is viewing a dynamic sequence of images. Our work is motivated by applications that involve gaze-contingent interactive displays on which information is displayed as a function of gaze direction. The approach therefore differs from standard approaches in two ways: (1) we deal with dynamic scenes, and (2) we provide means of adapting the model to a particular observer. As an indicator for the degree of saliency we evaluate the intrinsic dimension of the image sequence within a geometric approach implemented by using the structure tensor. Out of these candidate saliency-based locations, the currently attended location is selected according to a strategy found by supervised learning. The data are obtained with an eye-tracker and subjects who view video sequences. The selection algorithm receives candidate locations of current and past frames and a limited history of locations attended in the past. We use a linear mapping that is obtained by minimizing the quadratic difference between the predicted and the actually attended location by gradient descent. Being linear, the learned mapping can be quickly adapted to the individual observer.

The live-wire approach is a well-known algorithm based on a graph search to locate boundaries for image segmentation. We will extend the original cost function, which is solely based on finding strong edges, so that the approach can take a large variety of boundaries into account. The cost function adapts to the local characteristics of a boundary by analyzing a user-defined sample using a continuous wavelet decomposition. We will finally extend the approach into 3D in order to segment objects in volumetric data, e. g., from medical CT and MR scans.

A new approach to estimate the surface curvatures from 3D triangular mesh surfaces with Gaussian curvature's geometry interpretation is proposed in this work. Unlike previous work, the proposed method does not use local surface fitting, partial derivative computation, or oriented normal vector recovery. Instead, the Gaussian curvature is estimated at a vertex as the area of its small neighborhood under the Gaussian map divided by the area of that neighborhood. The proposed approach can handle vertices with the zero Gaussian curvature uniformly without localizing them as a separate process. The performance is further improved with the local Bezier curve approximation and subdivision. The effectiveness of the proposed approach for meshes with a large range of coarseness is demonstrated by experiments. The application of the proposed method to 3D surface segmentation and 3D mesh feature extraction is also discussed.

In this paper, we address the problem of 3d motion and structure estimation of 3d objects appearing in 2d images; namely, the estimation of the parameters of 3d motion, such as rotation and translation in 3d space, and the third dimension of certain objects that can be seen in a time sequence of 2d images. The study of the problem will be restricted to the case of non-rigid objects. The main problem with non-rigid motion is that it is not structured in a way that would allow its estimation with a few parameters, as in the case of rigid motion. That is why the main effort in most of the approaches is in representing deformable objects with suitable models that require only a small number of parameters for their description. In this work we perform a survey of algorithms proposed for the 3d motion and structure estimation of non-rigid objects, using the Finite Element Method. A classification of established algorithms is presented, with respect to the actual technique utilized and the applications for which they are most suitable.

This paper describes three varieties of realism that need to be considered in evaluating computer graphics images and defines the criteria that need to be met if each kind of realism is to be achieved. The paper introduces a conceptual framework for thinking about realism in images, and describes a set of research tools for measuring image realism and assessing its value in graphics applications.

An integral part of computer graphics, machine vision and human vision understanding is modeling how a surface reflects light. There is a substantial body of work on models describing surface reflectance ranging from purely matte to purely specular. One of the advantages of diffuse reflectance is that the color and the intensity of the reflected light are separable for most materials. Color is determined by the chromophores of the material, while intensity depends on the scene geometry. In specular highlights the color and the intensity of a specularity depend on both the geometry and the index of refraction of the material, which in turn is a function of wavelength. The graphics and vision communities often employ the following simplifying assumption when modeling specular highlights: For non-conductive materials the color of the specularity is the color of the light source. We will show that in most cases this assumption is violated. Theoretical analysis demonstrates that even for non-metallic surfaces the reflectivity ratio at specularities varies with both wavelength and angle of incidence. Furthermore, our experiments with a multispectral sensor clearly show that the deviation of the color of the specularities from the color of the incident light can be consistently measured.

The perceptual effects of changes of texture luminance either between the eyes or over time have been studied in several experiments and have led to a better comprehension of phenomenons such as sieve effect, binocular and monocular lustre and rivaldepth. In this paper, we propose an ecological model of glittering texture and analyze glitter perception in terms of variations of texture luminance and animation frequency, in dynamic illumination conditions. Our approach is based on randomly oriented mirrors that are computed according to the specular term of Phong's image rendering formula. The sparkling effect is thus correlated to the relative movements of the resulting textured object, the light array and the observer's point of view. The perceptual effect obtained with this model depends on several parameters: mirrors' density, the Phong specular exponent and the statistical properties of the mirrors' normal vectors. The ability to independently set these properties offers a way to explore a characterization space of glitter. A rating procedure provided a first approximation of the numerical values that lead to the best feeling of typical sparkling surfaces such as metallic paint, granite or sea shore.

A tone mapping algorithm for displaying high contrast scenes was designed on the basis of the results of experimental tests using human subjects. Systematic perceptual evaluation of several existing tone mapping techniques revealed that the most "natural" appearance was determined by the presence in the output image of detailed scenery features often made visible by limiting contrast and by properly reproducing brightness. Taking these results into account, we developed a system to produce images close to the ideal preference point for high dynamic range input image data. Of the algorithms that we tested, only the Retinex algorithm was capable of retrieving detailed scene features hidden in high luminance areas while still preserving a good contrast level. This paper presents changes made to Retinex algorithm for processing high dynamic range images, and a further integration of the Retinex with specialized tone mapping algorithms that enables the production of images that appear as similar as possible to the viewer's perception of actual scenes.

Studies have shown that observers' judgment of surface orientation and curvature is affected by the presence of surface texture pattern. However, the question of designing a texture pattern that does not hide the surace information nor does convey a misrepresentation of the surface remains unsolved. The answer to this question has important potential imapct across a wide range of visualization application. Molecular modeling and radiation therapy are among the many fields that are in need of accurately visualizing their data and could benefit from such methods. Over the past several years we have carried out a series of experiments to investigate the impact of various texture pattern characteristics on shape perception. In this paper we report the result of our most recent study. The task in this study was adjusting surface attitude probes under three different texture conditions and a control condition in which no texture was present. We later compare the performances of the subjects. The three texture conditions were: a doubly oriented texture in which approximately evenly spaced lines followed both of the principal directions, a singly oriented texture in which lines followed only the first principal direction, and a singly oriented line integral convolution. Over a series of 200 trials (4 texture conditions, 10 surface probe locations * five repeated measures) a total of five naive participants were asked to adjust a circular probe. The probes were randomly located on an arbitrary curved surface and its perpendicular extension appeared to be oriented in the direction of the surface normal. An analysis of the results showed that the performance was best in the two directional texture condition. Performances were further decreased in one directional and no texture conditions (in that order). The paper is organized as follows. In Section 1 we briefly describe the motivation for our work. In Section 2 we describe our experimental methods, including a brief summary of the process of the stimuli preparation and a detailed presentation of statistic analysis of our experimental results. In Section 3 we discuss the implications of our findings and in the last section we will talk about our future plans.

We develop spatially adaptive, low-level, color and spatial texture features based on perceptual principles about the processing of texture and color information. We then propose an algorithm that combines these features to obtain image segmentations that convey semantic information that can be used for content-based retrieval. Our focus is images of natural scenes. The color texture features are based on the estimation of spatially adaptive dominant colors, which on one hand, reflect the fact that the human visual system cannot simultaneously perceive a large number of colors, and on the other, the fact that image colors are spatially varying. The spatially adaptive dominant colors are obtained using a previously developed adaptive clustering algorithm for color segmentation. The spatial texture features are based on a steerable filter decomposition, which offers an efficient and flexible approximation of early processing in the human visual system. We use the local energy of the subband coefficients as a simple but effective characterization of spatial texture. A median filter is used to distinguish the energy due to region boundaries from the energy of the textures themselves. Texture feature estimation requires a finite neighborhood that limits spatial resolution, while color segmentation provides accurate and precise edge localization. By combining texture with color information, the proposed algorithm can obtain robust segmentations that are accurate and precise. The performance of the proposed algorithm is demonstrated in the domain of photographic images, including low resolution, degraded, and compressed images.

Current models of primary visual cortex (V1) include a linear filtering stage followed by a gain control mechanism that explains some of the nonlinear behavior of neurons. The nonlinear stage has been modeled as a divisive normalization in which each input linear response is half-rectified, squared and then divided by a weighted sum of half-rectified and squared linear responses in a certain neighborhood. Recently, Simoncelli and colleagues have suggested that this normalization reduces the statistical dependence of neuron responses. In this communication, we present an efficient implementation of these ideas as a practical image representation, and suggest some applications. The linear stage is implemented as a four-level orthogonal wavelet decomposition based on Daubechies filters, and the nonlinear normalization stage uses an improved version of Simoncelli's scheme. The normalization parameters are adapted to minimize statistical dependence between the output responses, so that the resulting representation consists of a set of statistically independent features or visual events. Since both linear and non-linear transforms applied can be inverted, this representation can be highly useful in different applications.

This paper presents a psychophysical study on the perception of image orientation. Some natural images are extremely difficult even for humans to orient correctly or may not even have a "correct" orientation; the study provides an upper bound for the performance of an automatic system. Discrepant detection rates based on only low-level cues have been reported, ranging from exceptionally high in earlier work to more reasonable in recent work. This study allows us to put the reported results in the correct perspective. In addition, the use of a large, carefully chosen image set that spans the "photo space" (in terms of occasions and subject matter) and extensive interaction with the human observers should reveal cues used by humans at various image resolutions. These can be used to design a robust automatic algorithm for orientation detection. A collection of 1000 images (mix of professional photos and consumer snapshots) is used in this study. Each image is examined by at least five observers and shown at varying resolutions. Object recognition is expected to be more difficult (impossible for some images) at the lowest resolution and easier as the resolution increases. At each resolution, observers are asked to indicate the image orientation, the level of confidence, and the cues they used to make the decision. This study suggests that for typical images, the upper bound on accuracy is close to 98% when using all available semantic cues from high-resolution images and 84% if only low-level vision features and coarse semantics from thumbnails are used. The study also shows that sky and people are the most useful and reliable among a number of important semantic cues.

The performance of content-based image retrieval using low-level visual content has largely been judged to be unsatisfactory. Perceived performance could probably be improved if retrieval were based on higher-level content. However, researchers have not been very successful in bridging what is now called the "semantic gap" between low-level content detectors and higher-level visual content. This paper describes a novel "top-down" approach to bridging this semantic gap. A list of primitive words (which we call "lexical basis functions") are selected from a lexicon of the English language, and are used to characterize the higher-level content of natural outdoor images. Visual similarity between pairs of images are then "computed" based on the degree of similarity between their respective word lists. These "computed" similarities are then shown to correlate with subjectively perceived similarities between pairs of images. This demonstrates that the chosen set of lexical basis functions is able to characterize the multidimensional content (and similarity) of these image pairs in a manner that parallels their subjectively perceived content (and similarity). If a retrieval system could be designed to automatically detect the visual content represented by these basis functions, it could compute a similarity measure that would correlate with human subjective similarity rankings.

In this research we are concerned with computer interfaces with which subjects navigate through maze simulations which are essentially buildings, with corridors and intersections, such as frequently encountered in computer games and simulations. We wish to determine if virtual reality interfaces introduce a performance enhancement that might be expected for display configurations which mimic natural perceptual experiences. We have experimented primarily with two display conditions for presentation of and navigation through the mazes. Subjects either view the maze on a desktop computer monitor, turning and moving within the maze with the mouse in a way that is similar to the configurations used in most first-person role playing computer games, or they viewed the maze from a standing position with a head-mounted display, being free to direct the view of the maze through body and head movements, and using the depression of a mouse button to effect movement in the direction that they were facing. Head-tracking was required for this latter condition. As expected there are striking individual differences in subjects’ abilities to learn to traverse the mazes. Across a variety of maze configuration parameters which significantly do influence performance, the results indicate that the virtual reality enhancements have no effect subjects' ability to learn the mazes, either as route knowledge or as cognitive maps.

It has been reported that the operation efficiency during teleoperation under stereoscopic video images is inferior to that of using the naked eye on the spot in real working environments. One of the reasons to explain this might be the smaller amount of information which is provided to the subject when working under stereoscopic images compared with that gotten by direct viewing in the real working environment. In our daily lives, we obtain information through various senses; the visual sense, acoustical sense, tactile sense and so on. The smaller amount of information causes a low working performance in the teleoperation. Some papers showed the improvement of operation efficiency when the subject got acoustic and/or tactile feedback as well as visual feedback in a simulated environment. We investigated to clarify the effects of force and acoustic feedback information with stereoscopic video images to subjects on the performance of the teleoperation using an actual robot. In experiment 1, we used the teleoperated system in which subjects could control the robot arm (the slave manipulator) to pick up cylindrical objects and to insert them in a hole by operating the input-and-tactile-information-display-system (the master manipulator). The results of experiment 1 showed that the force and acoustic information did not improve the working performance, and rather interfered with it. In experiment 2, we investigated the operation efficiency under conditions that force or acoustic feedback was given in the teleoperated work by eliminating contact information acquirable from vision. As a result of the experiment, force and acoustic information was not an effective way to shorten the completion time when the actual teleoperated robot was used.

Motion capture is one of the basic effects of a moving surround. To explore the existence of a motion capture in the stereodomain, we designed dynamic autostereograms with target-surround configuration. The images consisted of several horizontal lines of disparate disks with the central line of disks specified as a target. The surround was set in stereomotion by alternating between two disparities, while the target did not change in disparity, but suddenly disappeared in phase with one of the two surround depth-planes, reappearing in phase with the other surround plane. The target was vividly perceived as moving in depth despite its lack of any disparity change or even a paired location in the second of the two alternating frames. The motion capture hypothesis predicts that the target should be seen to move in synchrony and in the same direction as the surround. However, surprisingly, the data showed that the target was always perceived disappearing in a backward direction and reappearing in a forward direction irrespective of the surround direction, thus suggesting that the reported illusory depth-motion into a stationary target is an independent perceptual phenomenon that has no relation to the expected capture paradigm.

The staircase illusion, in which many staircase-like planes appear when a dot matrix pattern is tilted in depth and observed binocularly from the horizontal direction, is a visual phenomenon ever known in the field of psychology/psychophysics on binocular vision. We studied and formulated the mechanism that generated the phenomenon, and could visualize the whole shape of 3D staircases through a binocular vision model in computer. Our formulation was carried out, based on a hypothesis called "the nearest-neighbor rule." Moreover, we found two types of new staircase illusions induced by a vertical square-wave gratings pattern tilted around horizontal/vertical axis. One is a round-edged staircase illusion, and the other is a vertical strip type staircase illusion. We also constructed a new binocular vision model to formulate the mechanism, adopting the concept of horizontal horopter, and applied it into these new phenomena, and succeeded to visualize the whole shape of their 3D structures. Our computed and visualized results were very similar to those ones human observed in the real world. Our computer model not only verified the results of psychophysical experiments but also showed the possibility to discover/forecast new unknown phenomena that have not ever been referred in previous psychological trials.

Attending any conference on visual perception undoubtedly leaves one exposed to the work of Salvador Dali, whose extended phase of work exploring what he dubbed, “the paranoiac-critical method” is very popular as examples of multiple perceptions from conflicting input. While all visual art is intertwined with perceptual science, from convincing three-dimensional illusion during the Renaissance to the isolated visual illusions of Bridget Riley’s Op-Art, direct statements about perception are rarely uttered by the artists in recent times. However, there are still a number of artists working today whose work contains perceptual questions and exemplars that can be of interest to vision scientists and imaging engineers. This talk will start sampling from Op-Art, which is most directly related to psychophysical test stimuli and then will discuss “perceptual installations” from artists such as James Turrell’s, whose focus is often directly on natural light, with no distortions imposed by any capture or display apparatus. His work generally involves installations that use daylight and focus the viewer on its nuanced qualities, such as umbra, air particle interactions, and effects of light adaptation. He is one of the last artists to actively discuss perception. Next we discuss minimal art and electronic art, with video artist Nam June Paik discussing the “intentionally boring” art of minimalism. Another artist using installations is Sandy Skoglund, who creates environments of constant spectral albedo, with the exception of her human occupants. Tom Shannon also uses installations as his media to delve into 3D aspects of depth and perspective, but in an atomized fashion. Beginning with installation concepts, Calvin Collum then adds the restrictive viewpoint of photography to create initially confusing images where the pictorial content and depth features are independent (analogous to the work of Patrick Hughes). Andy Goldsworthy also combines photography with concepts of physical object manipulation inherent in installation, but takes the “installation” out of the gallery into the outdoor natural world. In doing so, he highlights perceptual interactions of scene spectra, color induction, and color constancy. The sculptor Shigeo Fukuda creates intentionally designed multiple-view sculptures, where the perceived subject matter of the sculpture changes with viewing angle. Lastly, Doug Aitken explores the interaction of the functional and emotional aspects of vision.

A kind of display with interesting new applications for indirect viewing is discussed in more detail. This line-display is especially suitable for indirect viewing with a very wide horizontal field of view, e.g. as needed for driving vehicles in darkness and bad atmospheric conditions, when used with appropriate video-sensors. The basic idea about the line-display is to divide a display into two components, a physical horizontal single light-emitting line, which then is scanned vertically into a virtual image. The line-display then makes it possible to install large, panoramic screens in confined spaces. Self-motion in the surrounding real 3D-world generates motion parallax, except from a zone straight ahead. Using motion parallax for good perception requires a large horizontal field of view. A small magnification or demagnification in a scene from indirect viewing, results in a quite dramatic change in perception. Scaling between the real world scene and the perceived display image should therefore be 1:1 to preserve correct angular speed for objects in the scene. In addition, the larger the viewing distance to the display is, the less visual conflicts will arise, e.g. from convergence versus accommodation.

Representations necessarily lose some of the visual information available in corresponding real-world scenes. This paper will discuss evaluations of the extent to which luminance contrast and visibility is preserved with three different methods for representing real-world scenes. Method one involves using psychophysical data from contrast charts to select the best print from among a density-varied series of photographic prints. The second and third methods involve extending the dynamic range of the representation by using High Dynamic Range Image (HDRI) techniques. HDRI's can be created by combining multiple overlapping exposures of a scene, or via computer simulation. In method two, algorithms are used to compress the luminance information in the HDRI into the luminance range available in the display, while preserving visible contrast as much as possible. The third method uses a wide-field, high-dynamic-range viewer to present an image with a much wider dynamic range than is available in a photographic print or a CRT display. Each method represents an improvement over simple photographic representation. In conjunction with appropriate instructions on how to interpret the images and the extent to which the images can be regarded as faithful, methods such as these can support practical decisions in visual design and reconstruction.

Several studies have suggested that translation and convergence in a linear trichromatic color space lead to the perception of transparency, but other transformations, such as shear and rotation, do not. We have designed a psychophysical experiment to study the limits of such systemic chromatic changes, adding categories such as different luminance levels and vector lengths. The number of our stimuli and the number of observations provide strong statistical support for D'Zmura's model. Our main results show that for vectors exceeding a minimal length, convergence and translation (except in the equiluminant plane) lead to the perception of transparency, while shear and divergence do not. However, our results reveal that small shears and divergences also appear transparent. We also found that large translations in the equiluminant plane tend to be less often judged as transparent.

Our previous experiments with additive and multiplicative transparent text on textured backgrounds show that readability can be more accurately predicted by adjusting the contrast with a contrast-gain-like divisive factor that includes the background RMS contrast. However, the factor performed poorly at predicting readability differences on two different patterned backgrounds. Using the same images of the previous study we presented the target words alone and single letters cut out of the target words. We found that word identification and word discriminability was affected by the backgrounds in the same way that the paragraph search performance was affected, but that letter identifiability on these two backgrounds was predicted by the metric. We also found a significant improvement from including different contrast gains for positive and negative contrasts in the metric. Unfortunately, word readability is not necessarily simply related to letter identifiability and simple contrast measures.

Extended Visual Cryptography is a method which encodes a number of images so that when the images are superimposed, the hidden image appears without a trace of original images. The decryption is done directly by human eyes without cryptographic calculations. The proposing system takes three pictures as input and generates two images which correspond to two of the input pictures. The third picture is perceived by superimposing the two output images. Previous methods are based on halftoning and Boolean operations. Transparency values must be quantized before encryption, and a pixel is halftoned by a fixed numbers of completely transparent and opaque subpixels. Then a transparency of the superimposed pixel is controlled by changing the subpixel arrangements of the two output pixels. Since the subpixel arrangement is basically random, a tradeoff exists that to express the more graylevels, each subpixel must become the smaller, making it the more difficult to superimpose by hand. Our new approach tolerates registration error for the third image and eases the difficulty, by adopting concentric-circular subpixel arrangement and continuous grayscale subpixel values. The system becomes considerably robust to the registration error. Also, it achieves quality improvement for all three images, by explicitly dealing with continuous graylevels.

The commercial success of color sequential displays is limited by the fact that people perceive multiple color images during pursuit and saccadic eye movements. We conducted a psychophysical experiment to quantify visibility of these color artifacts for different saccadic speeds, display background brightness, and target size. An Infocus sequential-color projector was placed behind a projection screen to simulate a normal desktop display. Saccadic eye movements were induced by requiring subjects to recognize text targets displayed at two different screen locations in rapid succession. The speed of saccadic movements was varied by manipulating the distance between the two target locations. A white bar, either with or without a yellow and red color fringe on the right edge, was displayed as subjects moved their eyes for the text recognition task. The two versions of the white bar will not be distinguishable if color break-up is present, thus performance of this task can be used as a measure of color break-up. The visibility of sequential color breakup decreases with background intensity and size of the white target, and increases with saccadic speed.

When one says “I see,” what is usually meant is “I understand.” But what does it mean to create a sense of understanding a large, complex, problem, one with many interlocking pieces, sometimes ill-fitting data and the occasional bit of contradictory information? The traditional computer science perspective on helping people towards understanding is to provide an armamentarium of tools and techniques - databases, query tools and a variety of graphing methods. As a field, we have an overly simple perspective on what it means to grapple with real information. In practice, people who try to make sense of some thing (say, the life sciences, the Middle East, the large scale structure of the universe, their taxes) are faced with a complex collection of information, some in easy-to-digest structured forms, but with many relevant parts scattered hither and yon, in forms and shapes too difficult to manage. To create an understanding, we find that people create representations of complex information. Yet using representations relies on fairly sophisticated perceptual practices. These practices are in no way preordained, but subject to the kinds of perceptual and cognitive phenomena we see in every day life. In order to understand our information environments, we need to learn to perceive these perceptual elements, and understand when they do, and do not, work to our advantage. A more powerful approach to the problem of supporting realistic sensemaking practice is to design information environments that accommodate both the world’s information realities and people’s cognitive characteristics. This paper argues that visual aspects of representation use often dominate sensemaking behavior, and illustrates this by showing three sensemaking tools we have built that take advantage of this property.

People with hearing loss make use of visual speech cues to supplement the impoverished speech signal. This process, known as speechreading (or lipreading) can be very effective because of the complementary nature of auditory and visual speech cues. Despite the importance of visual speech cues (for both normal-hearing and hearing-impaired people) research on the visual characteristics of speech has lagged behind research on the acoustic characteristics of speech. The field of acoustic phonetics benefited substantially from the availability of powerful techniques for acoustic signal analysis. The substantial, recent advances in optical signal processing have opened up new vistas for visual speech analysis analogous to the way technological innovation revolutionized the field of acoustic phonetics. This paper describes several experiments in the emerging field of optic phonetics.