The combination of Computer-Generated Holography (CGH) and deep learning has opened the possibility to generate both real-time and high-quality holograms. However, the widely-used data-driven deep learning method faces the problem of the large number of labeled training datasets generated by traditional algorithms, such as Gerchberg–Saxton (GS) iterative algorithm. It always takes a long time and limits the training performance of the network. In this work, we propose a model-driven neural network for high-fidelity Phase-Only Hologram (POH) generation. The Fresnel diffraction process is introduced as the physical model, which makes the network can automatically learn the latent encodings of POHs in an unsupervised way. Furthermore, the sub-pixel convolution upsampling method effectively improves the reconstruction quality. Once the training is completed, the POH of any two-dimensional image can be quickly generated. The calculation time is one to two orders of magnitude faster.
The coded aperture lensless imaging is a flexible system that replaces bulky lenses with a coded mask. The convolutional model expresses its forward imaging process compactly but fails when the measurements are distorted due to hardware restrictions. To overcome this problem, we generalize the convolutional model by introducing restricted factors to the imaging forward model explicitly. In detail, hardware restrictions are categorized into the linear part and the noise-like part. A compressed sensing algorithm based on the gradient sparsity of natural images is employed to solve the ill-posed inverse problem. Both numerical and experimental tests verify that the proposed model alleviates the artifact effect caused by down-sampling while performing reconstruction with higher resolution. This method is also valid for multi-wavelength and 3D imaging, lowering the using threshold of lensless cameras.
Minimally invasive endoscopes are indispensable in biomedicine. Coherent fiber bundles (CFB) enable lensless endoscopes. However, the aberration correction is challenging. Instead of involving bulky devices, deep neural networks (DNN) will be used. The novel approach uses speckles, which are decoded by DNN to retrieve the 3D object information. Besides this far-field approach, near-field CFB-based high-resolution imaging is promising for neurosurgery. However, the inherent honeycomb artifacts of CFB reduce the resolution. The inherent artifact is eliminated by DNN and high frequency information could be retrieved. Both methods have smart concepts in common, and both pave the way towards early recognition of diseases.
Fiber-based lensless endoscopy is powerful tool for minimally invasive tissue in clinical practice. However, the inherent honeycomb-artifact reduce the resolution and increases diagnosis difficulty. We proposed an end-to-end resolution enhancement and classification network for fiber bundle imaging. Comparing with conventional interpolation and filtering methods, the average peak signal to noise ratio (PSNR) can be improved 2~6 dB. Then we trained a VGG-19 classification network on label-free multiphoton images of 382 human braintumor 28 nontumor brain samples. The results show the classification accuracy of enhanced images is up to 91%, while the fiber bundle images are only 67% accurate. The method paves the way to in vivo histologic imaging through miniaturized endoscopic probes, and gives rapid and accurate determination for intraoperative diagnosis.
An autoencoder neural network is proposed for real-time phase-only CGH generation. As an unsupervised learning method, the input and output of the autoencoder are both the original images, which dispenses with calculating corresponding holograms. It could automatically learn the encoding of phase-only holograms during the training period. Once the training is completed, the phase-only hologram of any two-dimensional image can be quickly generated. The calculation time is 1-2 orders of magnitude faster than the traditional iterative algorithms and the reconstructed image quality is improved.
Digital holographic imaging is able to reconstruct 3D or phase information of the object from a one-shot 2D lensless hologram. The inverse reconstruction of 3D particle field could be realized based on the deep convolutional neural network. The hologram of a single particle is spread throughout the detector. Deep convolutional neural network could perform particle feature extraction and obtain the 3D position of each particle. We propose a learning-based approach for 3D holographic particle imaging. A dense encoder-decoder U-net network is designed. Compared with the CNN-based U-net network and the residual connection-based U-net network, the proposed network can reduce the number of network parameters, increase the amount of information of each layer of particles, extract accurate particle characteristics, and improve robustness. The Dense-U-net is more efficient in the way it processes data and requires a less memory storage for the learned model.
FZA imaging is one of the most promising lensless techniques because of its simple structure without calibration. The conventional image reconstruction methods are based on geometric optics model, then the error brought by diffraction degrades the imaging quality and limits the resolution. Here we quantitatively analyze the relationship between the width of FZA zone and the imaging resolution. The effects of illumination wavelength and the degree of coherence on image quality are also discussed. To improve the imaging quality, an image reconstruction method is proposed to minimize the influence of diffraction effect.
KEYWORDS: 3D modeling, 3D image processing, Detection and tracking algorithms, Virtual reality, Cameras, Cultural heritage, Augmented reality, Medical imaging, Volume rendering, Algorithm development
Three-dimensional (3D) model with high-quality texture is a powerful way to interact between virtual and reality 3D scene. It has significant applications in digital preservation of cultural heritage, virtual reality / augmented reality (VR/AR), medical imaging and other domains. High-quality texture reconstruction is one of the essential elements for 3D model. Due to the camera pose error and inaccurate model, current texture reconstruction methods could not eliminate the artifacts like blurring, ghosting and discontinuity.
In this work, multi-view high-definition images are captured for texture mapping. Normal-weight, depth-weight and edge-weight parameter are introduced to evaluate texture color confidence, respectively. The normalized weight factors are multiplied to generate a comprehensive weight parameter. By taking a weighted average of projected texture images, discontinuity of texture can be smoothed to a large extent. For large misalignment, bidirectional similarity (BDS) function, which represent the structural similarity between two images, are utilized to improve the texture image. The energy function is composed of two terms. One is Euclidean distance between target texture image and merged texture image. The other is BDS between target texture image and original texture image. By minimizing the energy function, the texture image could generate small local displacement while retaining the original structural information. The target and merged images are optimized alternately, the target image is calculated by patch-match algorithm, and the merged image is derived from weighted average of target images. The method we proposed could produce seamless texture comparing with Markov random field (MRF) algorithm. The definition could be higher than the camera parameter optimization algorithm. The experimental results show that structural similarity (SSIM) between the reconstructed images and the ground truth is higher than traditional algorithms. When dealing with inaccurate models with less than 10,000 facets, the SSIM value could be doubled compared with current algorithms.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.