The generation of a water window attosecond pulse with circular polarization is important for ultrafast research such as X-ray magnetic circular dichroism spectroscopy. In this paper, we propose a design method of broadband reflective quarter-wave plates in the water window region based on aperiodic Cr/Sc multilayer mirrors. Three broadband reflective quarter-wave plates are designed for achieving high circular polarization and circular reflection in the water window region with 10, 15, and 20 eV bandwidths. The results indicate that the reflected lights of the designed aperiodic Cr/Sc multilayer mirrors all show a more than 95% circular polarization degree and considerable circular reflection in their design band. The proposed broadband reflective quarter-wave plates can be used for controlling the polarization state of broadband water window region sources and generating the circularly polarized attosecond pulse by the linearly polarized water window region sources.