In this paper, we propose a method for single-pixel 2D/3D imaging to be applied in hyperspectral cameras operating in outdoor applications in unmanned aerial vehicles (UAVs). In rain rich environments, if camera systems operating in the visible range are used, alongside the atmospheric absorption that inevitably takes place, additional interactions occur between the light to be detected and the raindrops that produce Rayleigh and other scattering effects, thus limiting the visibility of the imagers mainly due to induced limitations in what radiation propagation is concerned. To solve this problem, we follow the approach that consists in using InGaAs based single photodetectors or line sensors that operate in the near-infrared (NIR) part of the spectra, in the wavelength range between 905 nm and 1600 nm. Due to the high costs of the few commercially available InGaAs based image sensors, we propose implementing a vision system based on the principle of single-pixel imaging (SPI) using active NIR illumination as a low-cost solution for image sensors in low-light scattering media. The SPI approach allows for the reconstruction of images from the electrical response signals of single InGaAs photodetectors to impinging active NIR illumination patterns created by an array of NIR LEDs. The latter is enabled by applying compression sensing (CS) algorithms. In this work, we aim at improving the quality of the reconstructed 2D images, defining the SSIM > 0.6 and RMSE<0.3 in rain rich scenarios as goal specifications. This SPI approach is additionally used to generate 3D images by applying Shape-From-Shading (SPS) and improving mesh 3D algorithms. By projecting Hadamard active illumination patterns analyzed in different regions of the reconstructed image, we enable scanning Spiral and Hilbert traces for image reconstruction to improve the SSIM level and reduce RMSE level of the reconstructed 2D/3D image obtained in the presence of rain. For testing purposes, we developed a controlled characterization system that enables rain simulation to evaluate the quality of the reconstructed 2D/3D single-pixel images.
KEYWORDS: Reconstruction algorithms, Image processing, Detection and tracking algorithms, Image restoration, Video acceleration, Video, Compressed sensing, Chemical species, Signal to noise ratio, Signal processing
The compressive sensing (CS) technique is a novel tool used to reconstruct images using fewer samples, normally sparse in the transform domain, than those required by conventional imaging systems. However, the methods applied for signal reconstruction within the CS approach still present some problems in the implementation, mainly due to their intensive computational demand and high power consumption requirements. These drawbacks need addressing if this approach is followed in systems aimed at e.g. drone autonomous flying or other embedded applications that additionally require very short processing times. In this paper we evaluate the use of hardware based parallel processing architecture for the implementation of the Orthogonal Matching Pursuit (OMP) algorithm, one of the most efficient CS reconstruction algorithms developed so far. To improve the algorithm performance, we target different maximum allowed processing times to reach minimum image resolutions required by each system of interest using different sparse (16 and 64) amounts of single-pixel generated samples per image. We also target the final image resolution to be above 20 dB in terms of the peak signal-to-noise ratio (PSNR). To reduce the execution and processing times required to generate each image, we propose implementing parallel kernels in the hardware platform for each of the operations required by the algorithms under study. In the proposed implementation the reconstructed images are used to generate video streams that form the foundation on which decisions are to be made by the system in continuous time, whereby each single image (frame) reconstruction cannot overcome 30 ms in order to maintain the minimum amount of frames per second (fps) above 33 (minimum required for an acceptable video stream). The implementation of a variation of the OMP algorithm in a graphics processing unit (GPU) using parallel architecture approach allows obtaining processing times 4 or 5 times shorter than those obtained if central processing unit (CPU) based architecture implementation is used for the same purpose.
During the last decades the radio detecting and ranging (RADAR) technology underwent an evolution transiting from the linear-frequency-modulated (LFM) systems developed in the 1970s, up to the orthogonal frequency-division multiplexing (OFDM) systems developed in the early 2000s. In mid 2010s, systems were proposed that combined the radar principle with optical solutions developed for imaging and ranging tasks following a hyperspectral embedded systems approach. The idea was to profit on the one side from the possibility offered by RADAR systems to work in harsh environments using emitted radio waves and detect mainly metal objects placed far away (hundreds of meters or even kilometers) from the detection system with positioning spatial resolutions in tens of centimeters, even if there are non-metallic barriers such as e.g. walls in between; and expand this possibility by using optical systems (e.g. light detecting and ranging –LIDAR- systems), using visible light active illumination, capable of generating 2D and 3D images of objects placed at much smaller distances from the detector, but allowing for much higher spatial resolutions (in the millimeter range). To reduce the atmospheric absorption of the emitted active illumination and increase the emitted optical power allowed for these systems that can correctly function even in harsh environments, we propose shifting the active illumination wavelengths from the visible range to the near infra-red (NIR) range, e.g. to 1550 nm. Lacking affordable image sensors fabricated in InGaAs technology, capable of detecting NIR radiation, in this paper we propose a hyperspectral imaging system using a very low power consuming single commercially available InGaAs photodiode to generate 2D images using the single-pixel imaging (SPI) approach based on compressive sensing (CS) and an array of NIR light emitting LEDs, combined with an 80 GHz millimeter band RADAR. The system is conceived to deliver a maximum radar range of 150 m with a maximum spatial resolution of ≤ 5 cm and a RADAR cross-section (RCS) of 10 – 50 m2, combined with an optical system capable of generating 24 fps video streams based on SPI generated images yielding a maximum ranging depth of 10 m with a spatial resolution of < 1 cm. The proposed system will be used in unmanned ground vehicle (UGV) applications enabling decision making in continuous time. The power consumption, dimensions and weight of the hyperspectral ranging system will be adjusted to the UGV targeted applications.
We present a case study for a time-of-flight (ToF) 3D imaging system using single-pixel imaging (SPI) approach based on compressive sensing (CS), accompanied by the Time-of-flight (ToF) principle applied to four reference points of the 2D image created and then mapped to the rest of the SPI generated virtual pixels. In this analysis we have developed a mathematical model of the system and evaluated three different scenarios considering different performance issues based on signal-to-noise ratio, different levels of background illumination, distance, spatial resolution, and material reflectivity presented by the objects in the scene. To be able to reduce the background photon shot noise and enable the correct functionality of the system also in harsh environments (in presence of micrometer size particles such as rain, snow, fog or smoke) we propose using near infra-red (NIR) active illumination with a peak wavelength of 1550 nm. The SPI principle is based here on an array of NIR emitting LEDs and the Thorlab FGA015 InGaAs single photodiode. For the system modelling and analysis, we considered the maximum background illumination intensity of up to 100 klux, different reflection coefficients of the target material to be detected, and measurement distances between 1 and 10 m. Using the ToF principle, we evaluated the direct ToF using both, pulsed laser NIR source as well as an array of NIR emitting LEDs combined with a single InGaAs photodiode on the one side, and an InGaAs single-photon avalanche diode (SPAD) on the other. Using the model developed, we estimated the spatial resolution (standard deviation from the distance measured) the proposed system might reach for each of the ToF methods analyzed and combining different system elements. Finally, we propose a SPI-ToF 3D imaging and ranging system for drone outdoors applications.
The Instituto de Astrofisica de Canarias (IAC) is undertaking the design and construction of a common-user near IR spectrograph (LIRIS) for the Cassegrain focus of the 4.2 m William Herschel Telescope sited at the Observatorio del Roque de Los Muchachos. LIRIS will be a near IR intermediate-resolution spectrograph designed to operate over a spectral resolution range between 1000 and 5000, with added capabilities for coronographic, multiproject and polarimetric observations. The instrument allows the combination of an adequate spatial resolution with a large useful field of view across the slit, thanks to the use of the new 1024 X 1024 pixel HgCdTe Hawaii detector manufactured by Rockwell. All the optics and mechanisms situated inside the cryostat will be cooled to below 100 K. The detector will operate at 77 K. Calibration and tracking will be made with the existing Cassegrain A and G Box, into which a near IR calibration system will be incorporated.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.