The purpose of low-light image enhancement is to improve image quality assessment by human visual perception and bolster the performance of subsequent visual tasks. It is necessary to consider not only the complexity of illumination in a real scene but also the issues of color distortion after image enhancement. We propose a two-stage network to solve these problems. In the first stage, we use a two-branch encoder–decoder subnetwork to learn multiscale features of the image, in which the brightness encoder branch is used to improve the network’s attention to a low-light region and improve the problem of uneven illumination. In the second stage, we utilize the detail recovery subnetwork to preserve details. In addition, we introduce an attention mechanism and cross-stage connection between the two stages to utilize the features learned by different subnetworks effectively. Extensive experiment results demonstrate that the proposed method outperforms several state-of-the-art methods in terms of quantitative metrics and the visual effect. Thus, our network enhances low-light images with high quality.