Skip to main content
Log in

Multi-Modal Image Registration Based on Local Self-Similarity and Bidirectional Matching

  • MATHEMATICAL THEORY OF IMAGES AND SIGNALS REPRESENTING, PROCESSING, ANALYSIS, RECOGNITION, AND UNDERSTANDING
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

The registration of images from infrared sensors and visible color sensors is a quite difficult problem due to their different phenomena. In this paper, we propose a new method to register visible and infrared images. The proposed approach consists of three main steps. In the first step, SURF (Speeded-Up Robust Features) algorithm is applied for local feature extraction. In the second step, LSS (local self-similarity descriptor) is computed for each extracted feature. Finally, a cross matching process followed by a consistency check in the projective transformation model is performed for feature correspondence and mismatch elimination. Experimental results show the proposed method achieves better accuracy for registering visible and infrared images as compared to state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.

Similar content being viewed by others

REFERENCES

  1. X. Liu, Y. Ai, J. Zhang, et al., “A novel affine and contrast invariant descriptor for infrared and visible image registration,” Remote Sens. 10 (4), 658–666 (2018).

    Article  Google Scholar 

  2. B. Zitova and J. Flusser, “Image registration methods: A survey,” Image Vision Comput. 21 (11), 977–1000 (2003).

    Article  Google Scholar 

  3. L. Gottesfeld Brown, “A survey of image registration techniques,” ACM Comput. Surv. 24 (4), 325–376 (1992).

    Article  Google Scholar 

  4. M. Deshmukh and U. Bhosle, “A survey of image registration,” Int. J. Image Process. 5 (3), 245–269 (2011).

    Google Scholar 

  5. J. Flusser and T. Suk, “A moment-based approach to registration of images with affine geometric distortion,” IEEE Trans. Geosci. Remote Sens. 32 (2), 382–387 (1994).

    Article  Google Scholar 

  6. K. Mikolajczyk and C. Schmid, “A performance evaluation of local descriptors,” IEEE Trans. Pattern Anal. Mach. Intell. 27 (10), 1615–1630 (2005).

    Article  Google Scholar 

  7. B. S. Reddy and B. N. Chatterji, “An FFT-based technique for translation, rotation, and scale-invariant image registration,” IEEE Trans. Image Process. 5 (8), 1266–1271 (1996).

    Article  Google Scholar 

  8. D. G. Lowe, “Object recognition from local scale-invariant features,” in Proceeding 7th of International Conference on Computer Vision (ICCV) (1999), pp. 1150–1157.

  9. H. Bay et al., “Speeded-up robust features (SURF),” Comput. Vision Image Understanding 110 (3), 346–359 (2008).

    Article  Google Scholar 

  10. E. Shechtman and M. Irani, “Matching local self-similarities across images and videos,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007), pp. 18–23.

  11. A. Sedaghat and H. Ebadi, “Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching,” ISPRS J. Photogramm. Remote Sens. 108, 62–71 (2015).

    Article  Google Scholar 

  12. P. Sourabh and U. C. Pati, “Optical-to-SAR image registration using modified distinctive order based self-similarity operator,” in 2018 IEEE International Students’ Conference on Electrical, Electronics, and Computer Science (SCEECS) (2018).

  13. B. Baheti et al., “A local self-similarity-based vehicle detection approach using single query image,” in 2nd International Conference on Computer Vision and Image Processing (2018), pp. 255–264.

  14. H. M. Chen, P. K. Varshney, and M. K. Arora, “Performance of mutual information similarity measure for registration of multitemporal remote sensing images,” IEEE Trans. Geosci. Remote Sens. 41 (11), 2445–2454 (2003).

    Article  Google Scholar 

  15. Y. S. Heo, K. M. Lee, and S. U. Lee, “Robust stereo matching using adaptive normalized cross-correlation,” IEEE Trans. Pattern Anal. Mach. Intell. 33 (4), 807–822 (2011).

    Article  Google Scholar 

  16. Z. Yi, C. Zhiguo, and X. Yang, “Multi-spectral remote image registration based on sift,” Electron. Lett. 44 (2), 107–108 (2008).

    Article  Google Scholar 

  17. A. Sedaghat, M. Mokhtarzade, and H. Ebadi, “Uniform robust scale-invariant feature matching for optical remote sensing images,” IEEE Trans. Geosci. Remote Sens. 49 (11), 4516–4527 (2011).

    Article  Google Scholar 

  18. J. Li and M. A. Nigel, “A comprehensive review of current local features for computer vision,” Neurocomputing 71 (10–12), 1771–1787 (2008).

    Article  Google Scholar 

  19. X. M. Zhang, X. H. Zhang, F. F. Ji, and Q. F. Yu., “Known landing area rough locating from far distance based on DDM-SIFT,” in IEEE Congress on Image and Signal Processing (2008), pp. 132–135.

  20. J. Dou, Q. Qin, Z. Tu, X. Peng, and Y. Li, “Infrared and visible image registration based on SIFT and sparse representation,” in 28th Chinese Control and Decision Conference (CCDC) (2016), pp. 5420–5424.

  21. X. Mei and H. Ling, “Robust visual tracking using l1 minimization,” in 12th International Conference on Computer Vision (ICCV) (2009), pp. 1–7.

  22. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, “Robust face recognition via sparse representation,” IEEE Trans. Pattern Anal. Mach. Intell. 31 (2), 210–227 (2009).

    Article  Google Scholar 

  23. G. Carneiro and A. D. Jepson, “Flexible spatial configuration of local image features,” IEEE Trans. Pattern Anal. Mach. Intell. 29 (12), 2089–2104 (2007).

    Article  Google Scholar 

  24. J. F. Dou and J. X. Li, “Automatic image mosaic based on SIFT using bidirectional matching,” Adv. Mater. Res. 457458, 841–847 (2012).

  25. S. Damotharasamy, “Approach to model human appearance based on sparse representation for human tracking in surveillance,” IET Image Process. 14 (11), 2383–2394 (2020).

  26. W. Zhong, H. Lu, and M. H. Yang, “Robust object tracking via sparsity-based collaborative model,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 1838–1845.

  27. J. Wang, J. Yang, K. Yu, et al., “Locality-constrained linear coding for image classification,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2010), pp. 3360–3367.

Download references

Funding

This work was supported by the by Shanghai University Outstanding Teachers Cultivation Fund Program A30DB1524011-21 and 2015 School Fund Project A01GY15GX48 and Shanghai Second Polytechnic University Mechanical Engineering Key Disciplines XXKZD1603 and the Construction of University Enterprise Cooperation Automobile Electronic Joint Experiment Center, grant no. A11NH182016.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to J. Dou, Q. Qin or Z. Tu.

Ethics declarations

The authors declare that they have no conflict of interest.

Additional information

Jian-fang Dou. He is currently a teacher with the School of Intelligent Manufacturing and Control Engineering, Shanghai Second Polytechnic University, Shanghai, China. In 2006, obtained the Bachelor Degree, Geography Information System, College of Traffic and Transport, Hebei Polytechnic University, Hebei Province, China. In 2009, obtained the Master Degree, Photogrammetry and Remote Sensing, School of Civil Engineering, Tongji University, Shanghai, China. In 2014, received the PhD Degree from Shanghai Jiaotong University, Shanghai, China. His major interests are object detection and tracking, machine learing, intelligent measurement and control.

Qin Qin. She is currently an Associate Professor with the School of Intelligent Manufacturing and Control Engineering, Shanghai Second Polytechnic University, Shanghai, China. In 2006, received the PhD Degree from the Shanghai Institute of Technical Physics of the Chinese Academy of Sciences, Shanghai, China. Her major interests are intelligent detection, image processing, and pattern recognition.

Zi-mei Tu. She is currently a teacher with the School of Intelligent Manufacturing and Control Engineering, Shanghai Second Polytechnic University, Shanghai, China. In 2012, obtained the Bachelor Degree, Information display and photoelectric technology, Shanghai Second Polytechnic University, Shanghai, China. In 2014, obtained the Master Degree, Shanghai Normal University, China. Her major interests are image segmentation and three-dimension reconstruction.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dou, J., Qin, Q. & Tu, Z. Multi-Modal Image Registration Based on Local Self-Similarity and Bidirectional Matching. Pattern Recognit. Image Anal. 31, 7–17 (2021). https://doi.org/10.1134/S1054661820040112

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661820040112

Keywords:

Navigation