Optical diffractive neural networks (ODNNs) implement a deep learning framework using passive diffractive layers. Although ODNNs offer unique advantages for light-speed, parallel processing, and low power consumption, their accuracy of image reconstruction still needs to be further improved. Here, we extend ODNNs to deep optics in lensless optics by proposing an optical-electronic neural network (OENN) for multi-modality encoder design, and high-accurate image reconstruction. The OENN includes an ODNN in which a pixel-level learnable diffractive layer is included for lensless camera design and an electrical convolutional neural network for image reconstruction. And the performance of OENN is relatively comparable. For speckle reconstruction using phase information, Pearson correlation coefficient (PCC) and peak signal-to-noise ratio (PSNR) can reach to 0.929 and 19.313 dB, respectively. For speckle reconstruction using intensity information, PCC and PSNR can reach to 0.955 and 19.779 dB, respectively. For lens imaging with phase information, PCC and PSNR can reach above 0.989 and 29.930 dB, respectively. For lens imaging with intensity information, PCC and PSNR can reach above 0.990 and 30.287 dB, respectively. In the future, the proposed optoelectronics artificial intelligent framework can be further applied for “end-to-end” optics design and imaging process of lensless or computational imaging modalities.