Remote sensing image processing has been widely used in environmental monitoring, terrain survey, military investigation, disaster early warning, and other fields. The remote sensing images matching is a key step, and the accuracy and real-time of the alignment have a great impact on these applications. Due to the high resolution and complexity of remote sensing images, scale-invariant feature transformation (SIFT) algorithm has the problems of high computational complexity and poor matching effect. Adaptive threshold adjustment algorithm has proposed in the remote sensing image matching, but minor changes in the contrast threshold can bring about drastic changes in the image matching quality, which will affect applications such as monitoring, measurement, and early warning. To improve the matching quality of a SIFT detector, an adaptive contrast threshold SIFT method based on image complexity calculation (CACT-SIFT) is proposed. The CACT-SIFT finds two contrast thresholds for the target image and the reference image, respectively, and achieves the target by minimizing the proposed criteria Cik. Experiments show that the method can be applied to the matching of remote sensing images. The information of reference images and target images can be detected at the same time, and the key points can be extracted in a robust way, with better accuracy and real-time accuracy of the alignment.