For local intensity-based image registration methods, a template of a predefined size is adopted. Some measure of similarity between the images is used to determine when the optimal alignment has occurred. Most intensity-based similarity utilizes the statistical and spatial information of an image. However, there are significant nonlinear radiometric differences between infrared and visible data. Images of natural scenes usually do not have enough spatial features within the template. Hence, many similarities will be useless when dealing with infrared and visible data. Minor eigenvalues (ME) image registration similarity is presented with the exploitation the spectral properties of remote sensing images. ME similarity is based on the linear spectral mixture model and detects control points through searching the minimum of ME of the covariance matrix. Experiments on Landsat-7 satellite Enhanced Thematic Mapper Plus data are performed to verify and evaluate the effectiveness. Transformation performance curves, correct match ratio (CMR), and registration accuracy are also discussed. According to the data, root-mean-square error of phase correlations is 0.0722 pixels and the CMR of ME similarity is nearly 100%. The results on the basis of TM1, TM2, TM3, and TM4 band images indicate that the proposed similarity holds promise for infrared and color image registration in natural scenes, with advantages over previous normal mutual information and gradient mutual information similarities.