Deep co-design methods have been proposed to optimize simultaneously optical and neural network parameters for many separate tasks such as high dynamic range, extended depth of field (EDOF), depth from defocus (DfD), object detection or pose estimation. In contrast, we study the multi-task co-design of an imaging system for two antagonist tasks: EDOF and DfD. We model and optimize a chromatic Cooke triplet using differentiable ray tracing, and we compare the performances for DfD and EDOF tasks, in a single, parallel and collaborative optimization scheme. We show how one task can benefit from the result of the other task. We also explore the benefit of the local positional information to process images with spatially varying point spread functions related to optical field aberrations.
KEYWORDS: Model based design, Data modeling, Mathematical optimization, Signal filtering, Tunable filters, Neural networks, Image processing, Deblurring, Process modeling
Co-design consists of optimizing the parameters of the lens and the processing together to obtain a gain in performance for the entire optical/processing chain, which requires new optimization tools, as traditional optical design ones can no longer be used easily. In the state of the art, joint optical/processing optimization methods based on statistical scene and blur models and analytical performance models have been proposed, and recently, new approaches based on the joint optimization of a neural network and optical components with a large database have been developed. However, to the best of our knowledge, no comparison of co-design results using either a model-based or a data-based approach for the same task have been conducted, which is the scope of this paper. We consider here the optimization of phase masks to extend the camera depth of field. We compare the optimization results using a performance model based on restoration error using either a generalized Wiener filter, or a neural network. We investigate the optimization trend depending on the neural network complexity, the starting point of the optimization and the possible interaction between the two approaches.
Co-design methods started to incorporate neural networks a few years ago when deep learning showed promising results in computer vision. This requires the computation of the point spread function (PSF) of an optical system as well as its gradients with respect to the optical parameters so that they can be optimized using gradient descent. In previous works, several approaches have been proposed to obtain the PSF, most notably using paraxial optics, Fourier optics or differential ray tracers. All these models have limitations and strengths regarding their ability to compute a precise PSF and their computational cost. We propose to compare them in a simple co-design task to discuss their relevance. We will discuss the computational cost of these methods as well as their applicability.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.