Structured illumination microscopy (SIM) has emerged as a powerful technique, surpassing the limitations imposed by optical diffraction and providing remarkable enhancements in both lateral and axial resolution compared with traditional diffraction-limited microscopy. However, it does come with certain limitations, including the need for a complex optical setup, extensive image acquisition, and computationally intensive post-processing. Motivated by the advancements in deep-learning-based super-resolution techniques, we propose an original three-dimensional (3D) representative learning algorithm called the transformer-based generative adversarial network (TransGAN), which can accurately predict corresponding aberrations through a combination of 17 mixed Zernike modes. Our approach outperforms state-of-the-art algorithms in various cellular structures, achieving impressive results with a mean square error of 2.358×10−5 for aberration determination. TransGAN presents a promising solution for enhancing SIM imaging, offering improved resolution and precise aberration estimation. This technique exhibits significant potential in overcoming the limitations associated with 3D SIM techniques and advancing the field of 3D optical microscopy.
Optical diffractive neural networks (ODNNs) implement a deep learning framework using passive diffractive layers. Although ODNNs offer unique advantages for light-speed, parallel processing, and low power consumption, their accuracy of image reconstruction still needs to be further improved. Here, we extend ODNNs to deep optics in lensless optics by proposing an optical-electronic neural network (OENN) for multi-modality encoder design, and high-accurate image reconstruction. The OENN includes an ODNN in which a pixel-level learnable diffractive layer is included for lensless camera design and an electrical convolutional neural network for image reconstruction. And the performance of OENN is relatively comparable. For speckle reconstruction using phase information, Pearson correlation coefficient (PCC) and peak signal-to-noise ratio (PSNR) can reach to 0.929 and 19.313 dB, respectively. For speckle reconstruction using intensity information, PCC and PSNR can reach to 0.955 and 19.779 dB, respectively. For lens imaging with phase information, PCC and PSNR can reach above 0.989 and 29.930 dB, respectively. For lens imaging with intensity information, PCC and PSNR can reach above 0.990 and 30.287 dB, respectively. In the future, the proposed optoelectronics artificial intelligent framework can be further applied for “end-to-end” optics design and imaging process of lensless or computational imaging modalities.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.