KEYWORDS: Cameras, Imaging systems, Sensors, Unmanned aerial vehicles, Control systems, System integration, Data processing, Visualization, Data acquisition, UAV imaging systems
Multispectral imaging technology analyzes for each pixel a wide spectrum of light and provides more spectral
information compared to traditional RGB images. Most current Unmanned Aerial Vehicles (UAV) camera systems are
limited by the number of spectral bands (≤10 bands) and are usually not fully integrated with the ground controller to
provide a live view of the spectral data.
We have developed a compact multispectral camera system which has two CMV2K 4x4 snapshot mosaic sensors
internally, providing 31 bands in total covering the visible and near-infrared spectral range (460-860nm). It is compatible
with (but not limited to) the DJI M600 and can be easily mounted to the drone. Our system is fully integrated with the
drone, providing stable and consistent communication between the flight controller, the drone/UAV, and our camera
payload. With our camera control application on an Android tablet connected to the flight controller, users can easily
control the camera system with a live view of the data and many useful information including histogram, sensor
temperature, etc. The system acquires images at a maximum framerate of 2x20 fps and saves them on an internal storage
of 1T Byte. The GPS data from the drone is logged with our system automatically. After the flight, data can be easily
transferred to an external hard disk. Then the data can be visualized and processed using our software into single
multispectral cubes and one stitched multispectral cube with a data quality report and a stitching report.
Recently, new programming paradigms have emerged that combine parallelism and numerical computations with algorithmic differentiation. This approach allows for the hybridization of neural network techniques for inverse imaging problems with more traditional methods such as wavelet-based sparsity modelling techniques. The benefits are twofold: on the one hand traditional methods with well-known properties can be integrated in neural networks, either as separate layers or tightly integrated in the network, on the other hand, parameters in traditional methods can be trained end-to-end from datasets in a neural network "fashion" (e.g., using Adagrad or Adam optimizers). In this paper, we explore these hybrid neural networks in the context of shearlet-based regularization for the purpose of biomedical image restoration. Due to the reduced number of parameters, this approach seems a promising strategy especially when dealing with small training data sets.
Many inverse problems (e.g., demosaicking, deblurring, denoising, image fusion, HDR synthesis) share various similarities:
degradation operators are often modeled by a specific data fitting function while image prior knowledge (e.g., sparsity)
is incorporated by additional regularization terms. In this paper, we investigate automatic algorithmic techniques for evaluating
proximal operators. These algorithmic techniques also enable efficient calculation of adjoints from linear operators
in a general matrix-free setting. In particular, we study the simultaneous-direction method of multipliers (SDMM) and the
parallel proximal algorithm (PPXA) solvers and show that the automatically derived implementations are well suited for
both single-GPU and multi-GPU processing. We demonstrate this approach for an Electron Microscopy (EM) deconvolution
problem.
In this work we present Liborg, a spatial mapping and localization system that is able to acquire 3D models on the y using data originated from lidar sensors. The novelty of this work is in the highly efficient way we deal with the tremendous amount of data to guarantee fast execution times while preserving sufficiently high accuracy. The proposed solution is based on a multi-resolution technique based on octrees. The paper discusses and evaluates the main benefits of our approach including its efficiency regarding building and updating the map and its compactness regarding compressing the map. In addition, the paper presents a working prototype consisting of a robot equipped with a Velodyne Lidar Puck (VLP-16) and controlled by a Raspberry Pi serving as an independent acquisition platform.
Realistic visualization is crucial for a more intuitive representation of complex data, medical imaging, simulation, and entertainment systems. In this respect, multiview autostereoscopic displays are a great step toward achieving the complete immersive user experience, although providing high-quality content for these types of displays is still a great challenge. Due to the different characteristics/settings of the cameras in the multiview setup and varying photometric characteristics of the objects in the scene, the same object may have a different appearance in the sequences acquired by the different cameras. Images representing views recorded using different cameras, in practice, have different local noise, color, and sharpness characteristics. View synthesis algorithms introduce artifacts due to errors in disparity estimation/bad occlusion handling or due to an erroneous warping function estimation. If the input multiview images are not of sufficient quality and have mismatching color and sharpness characteristics, these artifacts may become even more disturbing. Accordingly, the main goal of our method is to simultaneously perform multiview image sequence denoising, color correction, and the improvement of sharpness in slightly defocused regions. Results show that the proposed method significantly reduces the amount of the artifacts in multiview video sequences, resulting in a better visual experience.
While many existing CT noise filtering post-processing techniques optimize minimum mean squared error (MSE)-based quality metrics, it is well-known that the MSE is generally not related to the diagnostic quality of CT images. In medical image quality assessment, model observers (MOs) have been proposed for predicting diagnostic quality in medical images. MOs optimize a task-based quality criterion such as lesion or tumor detection performance. In this paper, we first discuss some of the non-stationary noise properties of CT noise. These properties will be utilized to construct a multi-directional non-stationary noise model that can be used by MOs. Next, we investigate a new shearlet-based denoising scheme that opti- mizes a task-based image quality metric for CT background noise. This work makes a connection between multi-resolution sparsity-based denoising techniques on the one hand and model observers on the other hand. The main advantage is that this approach avoids the two-step procedure of MSE-optimized denoising followed by a MO-based quality evaluation (of- ten with contradictory quality goals), while instead optimizing the desired task-based image quality directly. Experimental results are given to illustrate the benefits of the proposed approach.
Realistic visualization is crucial for more intuitive representation of complex data, medical imaging, simulation and entertainment systems. Multiview autostereoscopic displays are great step towards achieving complete immersive user experience. However, providing high quality content for this type of displays is still a great challenge. Due to the different characteristics/settings of the cameras in the multivew setup and varying photometric characteristics of the objects in the scene, the same object may have different appearance in the sequences acquired by the different cameras. Images representing views recorded using different cameras in practice have different local noise, color and sharpness characteristics. View synthesis algorithms introduce artefacts due to errors in disparity estimation/bad occlusion handling or due to erroneous warping function estimation. If the input multivew images are not of sufficient quality and have mismatching color and sharpness characteristics, these artifacts may become even more disturbing. The main goal of our method is to simultaneously perform multiview image sequence denoising, color correction and the improvement of sharpness in slightly blurred regions. Results show that the proposed method significantly reduces the amount of the artefacts in multiview video sequences resulting in a better visual experience.
The digital revolution has reached hospital operating rooms, giving rise to new opportunities such as tele-surgery and tele-collaboration. Applications such as minimally invasive and robotic surgery generate large video streams that demand gigabytes of storage and transmission capacity. While lossy data compression can offer large size reduction, high compression levels may significantly reduce image quality. In this study we assess the quality of compressed laparoscopic video using a subjective evaluation study and three objective measures. Test sequences were full High-Definition videos captures of four laparoscopic surgery procedures acquired on two camera types. Raw sequences were processed with H.264/AVC IPPP-CBR at four compression levels (19.5, 5.5, 2.8, and 1.8 Mbps). 16 non-experts and 9 laparoscopic surgeons evaluated the subjective quality and suitability for surgery (surgeons only) using Single Stimulus Continuous Quality Evaluation methodology. VQM, HDR-VDP-2, and PSNR objective measures were evaluated. The results suggest that laparoscopic video may be lossy compressed approximately 30 to 100 times (19.5 to 5.5 Mbps) without sacrificing perceived image quality, potentially enabling real-time streaming of surgical procedures even over wireless networks. Surgeons were sensitive to content but had large variances in quality scores, whereas non-experts judged all scenes similarly and over-estimated the quality of some sequences. There was high correlation between surgeons’ scores for quality and “suitability for surgery”. The objective measures had moderate to high correlation with subjective scores, especially when analyzed separately by camera type. Future studies should evaluate surgeons’ task performance to determine the clinical implications of conducting surgery with lossy compressed video.
In this paper, we first briefly review the directional properties of the Dual-Tree complex wavelet transform and we investigate how the directional selectivity of the transform can be increased (i.e., to obtain more than 6 orientations per scale). To this end, we describe a new augmented Lagrangian optimization algorithm to jointly perform the 2D spectral factorization of a set of 2D directional filters, with a high numerical accuracy. We demonstrate how this approach can be used to design compactly supported shearlet frames that are tight. Finally, a number of experimental results are given to show the merits of the resulting shearlet frames.
KEYWORDS: Signal detection, Image quality, Interference (communication), Image processing, Medical imaging, Magnetic resonance imaging, Signal processing, Signal to noise ratio, Statistical analysis, Tantalum
To improve imaging systems and image processing techniques, objective image quality assessment is essential. Model observers adopting a task-based quality assessment strategy by estimating signal detectability measures, have shown to be quite successful to this end. At the same time, costly and time-consuming human observer experiments can be avoided. However, optimizing images in terms of signal detectability alone, still allows a lot of freedom in terms of the imaging parameters. More specifically, fixing the signal detectability defines a manifold in the imaging parameter space on which different “possible” solutions reside. In this article, we present measures that can be used to distinguish these possible solutions from each other, in terms of image quality factors such as signal blur, noise and signal contrast. Our approach is based on an extended channelized joint observer (CJO) that simultaneously estimates the signal amplitude, scale and detectability. As an application, we use this technique to design k-space trajectories for MRI acquisition. Our technique allows to compare the different spiral trajectories in terms of blur, noise and contrast, even when the signal detectability is estimated to be equal.
In digital cameras and mobile phones, there is an ongoing trend to increase the image resolution, decrease the sensor size
and to use lower exposure times. Because smaller sensors inherently lead to more noise and a worse spatial resolution,
digital post-processing techniques are required to resolve many of the artifacts. Color filter arrays (CFAs), which use
alternating patterns of color filters, are very popular because of price and power consumption reasons. However, color
filter arrays require the use of a post-processing technique such as demosaicing to recover full resolution RGB images.
Recently, there has been some interest in techniques that jointly perform the demosaicing and denoising. This has the
advantage that the demosaicing and denoising can be performed optimally (e.g. in the MSE sense) for the considered noise
model, while avoiding artifacts introduced when using demosaicing and denoising sequentially.
In this paper, we will continue the research line of the wavelet-based demosaicing techniques. These approaches are
computationally simple and very suited for combination with denoising. Therefore, we will derive Bayesian Minimum
Squared Error (MMSE) joint demosaicing and denoising rules in the complex wavelet packet domain, taking local adaptivity
into account. As an image model, we will use Gaussian Scale Mixtures, thereby taking advantage of the directionality
of the complex wavelets. Our results show that this technique is well capable of reconstructing fine details in the image,
while removing all of the noise, at a relatively low computational cost. In particular, the complete reconstruction (including
color correction, white balancing etc) of a 12 megapixel RAW image takes 3.5 sec on a recent mid-range GPU.
In recent years, there has been a lot of interest in multiresolution representations that also perform a multidirectional analysis.
These representations often yield very sparse representation for multidimensional data. The shearlet representation,
which has been derived within the framework of composite wavelets, can be extended quite trivially from 2D to 3D.
However, the extension to 3D is not unique and consequently there are different implementations possible for the discrete
transform. In this paper, we investigate the properties of two relevant designs having different 3D frequency tilings. We
show that the first design has a redundancy factor of around 7, while in the second design the transform can attain a redundancy
factor around 3.5, independent of the number of analysis directions. Due to the low redundancy, the 3D shearlet
transform becomes a viable alternative to the 3D curvelet transform. Experimental results are provided to support these
findings.
The shearlet transform is a recent sibling in the family of geometric image representations that provides a traditional
multiresolution analysis combined with a multidirectional analysis. In this paper, we present a fast DFT-based analysis
and synthesis scheme for the 2D discrete shearlet transform. Our scheme conforms to the continuous shearlet theory to
high extent, provides perfect numerical reconstruction (up to floating point rounding errors) in a non-iterative scheme
and is highly suitable for parallel implementation (e.g. FPGA, GPU). We show that our discrete shearlet representation
is also a tight frame and the redundancy factor of the transform is around 2.6, independent of the number of analysis
directions. Experimental denoising results indicate that the transform performs the same or even better than several related
multiresolution transforms, while having a significantly lower redundancy factor.
This work explores the potentials of structure encoding in sparse tomographic reconstructions. We are encoding
spatial structure with Markov Random Field (MRF) models and employ it within Magnetic Resonance Imaging
(MRI) and Quantitative Microwave Tomography. We illustrate thereby also different ways of MRF modelling:
as a discrete, binary field imposed on hidden labels and as a continuous model imposed on the observable field.
In case of MRI, the analyzed approach is a straightforward extension of sparse MRI methods and is related
to the so-called LaMP (Lattice Matching Pursuit) algorithm, but with a number of differences. In case of
Microwave Tomography, we give another interpretation of structured sparsity using much different, but also
effective approach. Thorough experiments demonstrate clear advantages of MRF based structure encoding in
both cases and motivate strongly further development.
Traditional super-resolution methods produce a clean high-resolution image from several observed degraded low-resolution
images following an acquisition or degradation model. Such a model describes how each output pixel is related to one or
more input pixels and it is called data fidelity term in the regularization framework. Additionally, prior knowledge such
as piecewise smoothness can be incorporated to improve the image restoration result. The impact of an observed pixel on
the restored pixels is thus local according to the degradation model and the prior knowledge. Therefore, the traditional
methods only exploit the spatial redundancy in a local neighborhood and are therefore referred to as local methods.
Recently, non-local methods, which make use of similarities between image patches across the whole image, have
gained popularity in image restoration in general. In super-resolution literature they are often referred to as exemplarbased
methods. In this paper, we exploit the similarity of patches within the same scale (which is related to the class
of non-local methods) and across different resolution scales of the same image (which is also related to the fractal-based
methods). For patch fusion, we employ a kernel regression algorithm, which yields a blurry and noisy version of the
desired high-resolution image. For the final reconstruction step, we develop a novel restoration algorithm. The joint
deconvolution/denoising algorithm is based on the split Bregman iterations and, as prior knowledge, the algorithm exploits
the sparsity of the image in the shearlet-transformed domain. Initial results indicate an improvement over both classical
local and state-of-the art non-local super-resolution methods.
KEYWORDS: Denoising, Image quality, Image processing, Visualization, Image denoising, Wavelets, Super resolution, Digital image processing, 3D image processing, Video
In this paper we propose several improvements to the original non-local means algorithm introduced by Buades
et al. which obtains state-of-the-art denoising results. The strength of this algorithm is to exploit the repetitive
character of the image in order to denoise the image unlike conventional denoising algorithms, which typically
operate in a local neighbourhood. Due to the enormous amount of weight computations, the original algorithm
has a high computational cost.
An improvement of image quality towards the original algorithm is to ignore the contributions from dissimilar
windows. Even though their weights are very small at first sight, the new estimated pixel value can be severely
biased due to the many small contributions. This bad influence of dissimilar windows can be eliminated by setting
their corresponding weights to zero. Using the preclassification based on the first three statistical moments, only
contributions from similar neighborhoods are computed. To decide whether a window is similar or dissimilar,
we will derive thresholds for images corrupted with additive white Gaussian noise. Our accelerated approach is
further optimized by taking advantage of the symmetry in the weights, which roughly halves the computation
time, and by using a lookup table to speed up the weight computations. Compared to the original algorithm,
our proposed method produces images with increased PSNR and better visual performance in less computation
time.
Our proposed method even outperforms state-of-the-art wavelet denoising techniques in both visual quality
and PSNR values for images containing a lot of repetitive structures such as textures: the denoised images are
much sharper and contain less artifacts. The proposed optimizations can also be applied in other image processing
tasks which employ the concept of repetitive structures such as intra-frame super-resolution or detection of digital
image forgery.
In this paper we describe a novel approach to image interpolation while preserving sharp edge information.
Many interpolation methods already have been proposed in the literature, but suffer from one or more artifacts
such as aliasing, blurring, ringing etc. Non-linear methods or edge-directed methods result in sharp interpolated
images but often look segmented or have great visual degradation in fine structured textures. We concentrate in
this paper on tackling blurred edges by mapping the image's level curves. Image's level curves or isophotes are
spatial curves with a constant intensity level. The mapping of these intensity levels can be seen as a local contrast
enhancement problem, therefore we can rely on some contrast enhancement techniques. A great advantage of this
approach is that the shape of the level curves (and of the objects) are preserved and no explicit edge detection is
needed here. Additional constraints in function of the image interpolation are defined in this flexible framework.
Different strategies of extending greyscale interpolation to colour images are also discussed in this paper. The
results show a large improvement in visual quality: the edges are sharper and ringing effects are removed.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.