Skip to main content
SpringerLink
Log in

Additive margin cosine loss for image registration

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In view of the multi-scale changes and the influence of light and angle in the image matching process, it is quite difficult to realize intelligent image registration by using convolutional neural network. The existing image matching algorithm has the following problems in the application process: the existing shallow feature extraction model has lost a lot of effective feature information and low recognition accuracy. Meanwhile, the image registration method based on deep learning is not robust and accurate enough. Therefore, an image registration method based on additive edge cosine loss was proposed in this paper. In the twin network architecture, cosine loss was used to convert Euclidean space into angular space, which eliminated the influence of characteristic intensity and improved the accuracy of registration. The matching cost was directly calculated by the included angle of two vectors in the embedded space, where the size of the angle edge could be quantitatively adjusted through parameter \(m\). We further derived a specific \(m\) to quantitatively adjust the loss. In addition, anti-rotation attention mechanism was added to the network to enhance the ability of feature information extraction and adjust the position information of feature vectors to reduce the mismatching caused by image rotation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference (AVC), pp. 147–151 (1988)

  2. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)

    Article  Google Scholar 

  3. Yan, K., Rahul, S.: PCA-SIFT: a more distinctive representation for local image descriptors. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 506–513 (2004)

  4. Bay, H., Tuytelaars, T., Gool, L.: SURF: speeded up robust features. In: Proceedings of the 9th European Conference on Computer Vision (ECCV). Springer, Graz, Austria, pp. 404–417 (2006)

  5. Fan, B., Wu, F.C., Hu, Z.Y.: Line matching leveraged by point correspondences. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 390–397 (2010)

  6. Fan, B., Wu, F.C., Hu, Z.Y.: Robust line matching through line-point invariants. Pattern Recogn. 45(2), 794–805 (2012)

    Article  Google Scholar 

  7. Wang, Z.H., Wu, F.C., Hu, Z.Y.: MSLD: a robust descriptor for line matching. Pattern Recogn. 42(5), 941–953 (2009)

    Article  Google Scholar 

  8. Feng, R., Du, Q., Li, X., Shen, H.: Robust registration for remote sensing images by combining and localizing feature- and area-based methods. ISPRS J. Photogramm. Remote. Sens. 151(1), 15–26 (2019)

    Article  Google Scholar 

  9. Elnemr, H.A.: Combining SURF and MSER along with color features for image retrieval system based on bag of visual words. J. Comput. Sci. 12(4), 213–222 (2016)

    Article  Google Scholar 

  10. Zhang, X., Felix, X., Karaman, S., Chang, S.F.: Learning discriminative and transformation co-variant local feature detectors. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 281–591 (2017)

  11. Savinov, N., Seki, A., Ladicky, L., Sattler, T., Polle-feys, M.: Quad-networks: unsupervised learning to rank for interest point detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 218–223 (2017)

  12. Laguna, AB., Riba, E., Ponsa, D., Mikolajczyk, K. T.: Keypoint detection by hand-crafted and learned CNN filters. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 593–607 (2019)

  13. MooYi, K., Trulls, E., Lepetit, V., Fua, P.: Learned invariant feature transform (ECCV), pp. 319–328 (2016)

  14. Noh, H., Araujo, A., Sim, J., Weyandand, T.B., Han, B.: Large-scale image retrieval with attentive deep local features (ICCV), pp. 1612–1623 (2017)

  15. Daniel, D., Tomasz, M., Andrew, R.: Superpoint: self-supervised interest point detection and description. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–12 (2018)

  16. Yuki, O., Eduard, F., Pascal, F., Yi, K.M.: Learning local features from images. In: The Conference and Workshop on Neural Information Processing Systems (NeurIPS), pp. 125–137 (2018)

  17. Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., Sattler, T.: D2-Net: A trainable CNN for joint detection and description of local features. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1023–1030 (2019)

  18. Revaud, J., Weinzaepfel, P., Roberto C., Souza, D., Pion, N., Csurka, G., Cabon, Y., Humenberger, M.: R2D2: repeatable and reliable detector and descriptor. In: The Conference and Workshop on Neural Information Processing Systems (NeurIPS), pp. 1321–1335 (2019)

  19. Sdika, M.: A fast nonrigid image registration with constraints on the Jacobian using large scale constrained optimization. IEEE Trans. Med. Imaging 27(2), 71–81 (2008)

    Article  Google Scholar 

  20. Duchenne, O., Joulin, A., Ponce, J.: A graph-matching kernel for object categorization. In: IEEE International Conference on Computer Vision (ICCV), pp. 6–13 (2011)

  21. Ham, B., Cho, M., Schmid, C., Ponce, J.: Proposal flow. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1063–1072 (2016)

  22. Long, J.L., Zhang, N., Darrell, T.: Do convents learn correspondence? In: NIPS, pp. 1056–1067 (2014)

  23. Han, X., Leung, T., Jia, Y., Sukthankar, R., Berg, A.C.: MatchNet: Unifying feature and metric learning for patch-based matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3279–3286 (2015)

  24. Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4353–4361 (2015)

  25. Tian, Y.R., Fan, B., Wu, F.C.: L2-Net: deep learning of discriminative patch descriptor in Euclidean space. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6128–6136 (2017)

  26. Khan, M.J., Yousaf, A., Javed, N., et al.: Automatic target detection in satellite images using deep learning. J. Space Technol. 7(1), 44–49 (2017)

    Google Scholar 

  27. Haskins, G., Kruger, U., Yan, P., et al.: Deep learning in medical image registration: a survey. Mach. Vis. Appl. 31(1), 1–8 (2020)

    Google Scholar 

  28. Estienne, T., Lerousseau, M., Vakalopoulou, M., et al.: Deep learning-based concurrent brain registration and tumor segmentation. Front. Comput. Neurosci. 12(1), 256–267 (2020)

    Google Scholar 

  29. Khan, M.J., Yousaf, A., Khurshid, K., et al.: Automated forgery detection in multispectral document images using fuzzy clustering. In: IAPR International Workshop on Document Analysis Systems (DAS), pp. 1–7 (2018)

  30. Khan, M.J., Yousaf, A., Abbas, A., Khurshid, K.: Deep learning for automated forgery detection in hyperspectral document images. J. Electron. Imaging 27(5), 1110–1121 (2018)

    Article  Google Scholar 

  31. Yousaf, A., Khan, M.J., Siddiqui, A.M., et al.: A robust and efficient convolutional deep learning framework for age-invariant face recognition. Expert. Syst. 37(3), 503–512 (2020)

    Article  Google Scholar 

  32. Lei, Y., Fu, Y.B., Wang, T.H., et al.: 4D-CT deformable image registration using multiscale unsupervised deep learning. Phys. Med. Biol. 65(8), 1008–1021 (2020)

    Article  Google Scholar 

  33. Eppenhof, K., Lafarge, M.W., Veta, M., et al.: Progressively trained convolutional neural networks for deformable image registration. IEEE Trans. Med. Imaging 39(5), 1594–1604 (2019)

    Article  Google Scholar 

  34. Chen, S., Zhong, S., Xue, B., et al.: Iterative scale-invariant feature transform for remote sensing image registration. IEEE Trans. Geosci. Remote Sens. 99(1), 1–22 (2020)

    Google Scholar 

  35. Khan, M.J., Khan, H.S., Yousaf, A., et al.: Modern trends in hyperspectral image analysis: a review. IEEE Access 6(1), 14118–14129 (2018)

    Article  Google Scholar 

  36. Khan, M.J., Khurshid, K., Shafait, F.: A spatio-spectral hybrid convolutional architecture for hyperspectral document authentication. In: IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 1–8 (2019)

  37. Yang, T.Y., Hsu, J.H., Lin, Y.Y.: DeepCD: learning deep complementary descriptors for patch representations. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3334–3342 (2017)

  38. Balntas, V., Johns, E., Tang, L.L.: PN-Net: conjoined triple deep network for learning local image descriptors. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3129–3142 (2016)

  39. Altwaijry, H., Trulls, E., Hays, J., Fua, P., Belongie, S.: Learning to match aerial images with deep attentive architectures. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3539–3547 (2016)

  40. Balakrishnan, G., Zhao, A., Sabuncu, M., Gut-tag, J., Dalca, A., Oxelmorph, V.: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 9(5), 597–610 (2019)

    Google Scholar 

  41. Szegedy, C., Toshev, A., Erhan, D.: Deep neural networks for object detection. In: Annual Conference on Neural Information Processing Systems (NIPS), MIT Press, pp. 118–126 (2013)

  42. He, K., Sun, J.: Convolutional neural networks at constrained time cost. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5353–5360 (2015)

  43. Balakrishnan, G., Zhao, A., Sabuncu, MR., Gut-tag, J., Dalca, A.: An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9252–9260 (2018)

  44. Li, C., Sanchez, R.V., Zurita, G., Cerrada, M., Cabrera, D.: Fault diagnosis for rotating machinery using vibration measurement deep statistical feature learning. Sensors 16(6), 895–906 (2016)

    Article  Google Scholar 

  45. Mishchuk, A., Mishkin, D., Radenovic, F., Matas, J.: Working hard to know y our neighbor’s margins: local descriptor learning loss. In: The Conference and Workshop on Neural Information Processing Systems (NeurIPS), pp. 53–60 (2017)

  46. Kumar, V., Carneiro, G., Reid, I.: Learn-ing local image descriptors with deep siamese and triplet convolutional networks by minimising global loss functions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1740–1752 (2016)

  47. Tian, Y., Yu, X., Fan, B., Wu, F., Heijnen, H., Balntas, V.: SOSNet: second order similarity regularization for local descriptor learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 465–470 (2019)

  48. Ebel, P., Mishchuk, A., Kwang, M., Fua, P., Trulls. E.: Beyond Cartesian representations for local descriptors. In: The IEEE International Conference on Computer Vision (ICCV), pp. 24–38 (2019)

  49. Rocco, I., Arandjelovi, R., Sivic, J.: Convolutional neural network architecture for geometric matching. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3653–3675 (2017)

  50. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(1), 91–110 (2004)

    Article  Google Scholar 

  51. Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: The IEEE International Conference on Computer Vision (ICCV), pp. 1470–1479 (2003)

  52. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015)

  53. Hadsell, R., Chopra, S., Lecun, Y.: Dimensionality reduction by learning an invariant mapping, In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1735–1742 (2006)

  54. Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: International Conference on Neural Information Processing Systems (NeurIPS), pp. 1988–1996 (2014)

  55. Liu, W., Wen, Y., Yu, Z., Yang, M.: Large-margin softmax loss for convolutional neural networks. In: The International Conference on Machine Learning (ICML), pp. 128-137 (2016)

  56. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: SphereFace: deep hypersphere embedding for face recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1573–1585 (2017)

  57. Zheng, Z., Zheng, L., Yang, Y.: A discriminatively learned CNN embedding for person re-identification. ACM Trans. Multimed. Comput. Commun. Appl. 14(1), 1301–1320 (2017)

    Google Scholar 

  58. Dong, X., Yan, Y., Tan, M., Yang, Y., Tsang, I.W.: Late fusion via subspace search with consistency preservation. IEEE Trans. Image Process. 28(1), 518–528 (2019)

    Article  MathSciNet  Google Scholar 

  59. Zhong, D., Zhu, J.: Centralized large margin cosine loss for open-set deep palmprint recognition. IEEE Trans. Circuits Syst. Video Technol. 99(1), 283–290 (2019)

    Google Scholar 

  60. Bromley, J., Guyon, I., Lecun, Y., Sackinger, E., Shah, R.: Signature verification using a ‘Siamese’ time delay neural network. Int. J. Pattern Recognit. Artif. Intell. 7(4), 669–688 (1993)

    Article  Google Scholar 

  61. Yang, M., Liu, Y.G., You, Z.: The Euclidean embedding learning based on convolutional neural network for stereo matching. Neurocomputing 6(7), 132–148 (2017)

    Google Scholar 

  62. Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) European Conference on Computer Vision (ECCV), pp. 499–515. Springer, New York (2016)

    Google Scholar 

  63. Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A comprehensive study on center loss for deep face recognition. Int. J. Comput. Vis. 11(17), 668–683 (2019)

    Article  Google Scholar 

  64. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 99–107 (2017)

    Google Scholar 

  65. Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1492–1500 (2017)

  66. He, K., Zhang, Ren, S.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

  67. Bruzzone, L., Roli, F., Serpico, S.B.: An extension of the Jeffreys–Matusita distance to multiclass cases for feature selection. IEEE Trans. Geosci. Remote Sens. 33(9), 1318–1321 (1995)

    Article  Google Scholar 

  68. Lance, G.N., Williams, W.T.: Computer programs for hierarchical polythetic classification. Comput. J. 9(1), 60–64 (1966)

    Article  Google Scholar 

Download references

Funding

This research was funded by the National Natural Science Foundation of China, Grant Nos. 11664005; Science and technology planning project of Guizhou province, Grant No. 2020-1Y021; Postgraduate Education Innovation Plan of Guizhou Province, Grant No. YJSCXJH 2019-066; School-level project of Guizhou University of Finance and Economics in 2020, No. 2020XJC03.

Author information

Authors and Affiliations

  1. School of Information, Guizhou University of Finance and Economics, Guiyang, Guizhou, China

    Yuandong Ma, Shouyu Sun, Fengjiao Wu, Yunfan Yang, Xin Yang & Zijiang Luo

  2. Beijing Interjoy Technology Co., Ltd, Beijing, China

    Yuandong Ma, Shouyu Sun, Fengjiao Wu & Bin Xu

Authors
  1. Yuandong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shouyu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fengjiao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunfan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zijiang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zijiang Luo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, Y., Sun, S., Wu, F. et al. Additive margin cosine loss for image registration. Vis Comput 38, 1787–1802 (2022). https://doi.org/10.1007/s00371-021-02105-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02105-6

Keywords

Search

Navigation