Skip to main content
SpringerLink
Log in

A deep learning framework for face verification without alignment

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Most of the CNN (convolutional neural networks) methods require alignment, which will affect the efficiency of verification. This paper proposes a deep face verification framework without alignment. First and foremost, the framework consists of two training stages and one testing stage. In the first training stage, the CNN is fully trained on the large face dataset. In the second training stage, embedding triplet is adopted to fine-tune the models. Furthermore, in the testing stage, SIFT (scale invariant feature transform) descriptors are extracted from intermediate pooling results for cascading verification, which effectively improves the accuracy of face verification without alignment. Last but not least, two CNN architectures are designed for different scenarios. The CNN1 (convolutional neural networks 1), with fewer layers and parameters, requires a small amount of memory and computation in training and testing, so it is suitable for real-time system. The CNN2 (convolutional neural networks 2), with more layers and parameters, has excellent face verification. Through the long-term training on WEB-face dataset and experiments on the LFW (labled faces in the wild), YTB (YouTube) datasets, the results show that the proposed method has superior performance compared with some state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Reference

  1. Nanjun, H., et al.: Feature extraction with multiscale covariance maps for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 57, 1–15 (2019)

    Article  Google Scholar 

  2. Zhou, L., et al.: Combining multi-wavelet and CNN for palmprint recognition against noise and misalignment. IET Image Process. 13(9), 1470–1478 (2019)

    Article  Google Scholar 

  3. He, K. et al.: Identity mappings in deep residual networks. In: European Conference on Computer Vision Springer International Publishing, pp 630–645 (2016)

  4. Li, G., Yu, Y.: Visual saliency detection based on multiscale deep CNN features. IEEE Trans. Image Process. 25(11), 5012–5024 (2016)

    Article  MathSciNet  Google Scholar 

  5. Wang, W., et al.: Development of convolutional neural network and its application in image classification: a survey. Opt. Eng. 58(4), 1 (2019)

    Google Scholar 

  6. Li, Y., et al.: Learning a bi-level adversarial network with global and local perception for makeup-invariant face verification. Pattern Recogn. 90, 99–108 (2019)

    Article  Google Scholar 

  7. Sun, Y., Wang, X., Tang, A.: Deeply learned face representations are sparse, selective, and robust. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp 2892–2900 (2015)

  8. Wen, Y. et al.: A discriminative feature learning approach for deep face recognition. In: European Conference on Computer Vision Springer, pp 499–515 (2016)

  9. Chen, J.C., Patel, V.M., Chellappa, R.: Unconstrained face verification using deep CNN features. Applications of Computer Vision IEEE, pp 1–9 (2016)

  10. Liu, J., et al.: Targeting ultimate accuracy: face recognition via deep embedding. Comput. Res. Repos. 1506, 7310 (2015)

    Google Scholar 

  11. Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference, pp 41.1–41.12 (2015)

  12. Ding, H., Zhou, S.K., Chellappa, R.: Facenet2expnet: regularizing a deep face recognition net for expression recognition. In: 2017 12th IEEE international conference on automatic face and gesture recognition, pp 118–126 (2017).

  13. Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp 1891–1898 (2014)

  14. Ahmed, M., Serestina, V.: Deep learning using Bayesian optimization for facial age estimation. In: International Conference on Image Analysis and Recognition. Springer, Cham (2019)

  15. Ali, A. et al.: BioMetricNet: deep unconstrained face verification through learning of metrics regularized onto Gaussian distributions. arXiv preprint arXiv:2008.06021 (2020)

  16. Crosswhite, N., Byrne, J., Stauffer, C., Parkhi, O., Cao, Q., Zisserman, A.: Template adaptation for face verification and identification, vol. 79, pp. 35–48. Elsevier, New York (2018)

    Google Scholar 

  17. Zheng, Y., Pal, D.K., Savvides, M.: Ring loss: convex feature normalization for face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5089–5097 (2018)

  18. Deng, J. et al.: Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4690–4699 (2019)

  19. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Web-scale training for face identification. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp 2746–2754 (2015)

  20. Yi, D., Lei, Z., Liao, S., Li, SZ.: Learning face representation from scratch. arXiv preprint :1411.7923(2014)

  21. Shabat, A.M.M., Jules-Raymond, T.: Angled local directional pattern for texture analysis with an application to facial expression recognition. IET Comp. Vis. 12(5), 603–608 (2018)

    Article  Google Scholar 

  22. Cao, X. et al.: A practical transfer learning algorithm for face verification. In: IEEE International Conference on Computer Vision IEEE, pp 3208–3215 (2014)

  23. Choi, J.Y., Lee, B.: Ensemble of deep convolutional neural networks with Gabor face representations for face recognition. IEEE Trans. Image Process. 29, 3270–3281 (2019)

    Article  Google Scholar 

  24. Chen, D. et al.: Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3025–3032 (2013)

  25. Lin, G., et al.: Robust, discriminative and comprehensive dictionary learning for face recognition. Pattern Recogn. 81, 341–356 (2018)

    Article  Google Scholar 

  26. Chen, Y.C. et al.: Adaptive representations for video-based face recognition across pose. In: Applications of Computer Vision IEEE, pp 984–991 (2014)

  27. Fan, Z., et al.: Virtual dictionary based kernel sparse representation for face recognition. Pattern Recogn. 76, 1–13 (2018)

    Article  Google Scholar 

  28. Martínez-Díaz, Y., et al.: On fisher vector encoding of binary features for video face recognition. J. Vis. Commun. Image Represent. 51, 155–161 (2018)

    Article  Google Scholar 

  29. Parkhi, O.M. et al.: A compact and discriminative face track descriptor. In: IEEE Conference on Computer Vision and Pattern Recognition IEEE Computer Society, pp 1693–1700 (2014)

  30. Chen, J.C., Patel, V., Chellappa, R.: Landmark-based fisher vector representation for video-based face verification. In: IEEE International Conference on Image Processing IEEE, pp 2705–2709 (2015)

  31. Lu, J., et al.: Joint feature learning for face recognition. IEEE Trans. Inform. Foren. Secur. 10(7), 1371–1383 (2015)

    Article  Google Scholar 

  32. Huang, Z., et al.: Cross Euclidean-to-Riemannian metric learning with application to face recognition from video. IEEE Trans. Pattern Anal. Mach. Intell. 40(12), 2827–2840 (2017)

    Article  Google Scholar 

  33. Zhe, X., Chen, S., Yan, H.: Directional statistics-based deep metric learning for image classification and retrieval. Pattern Recogn. 93, 113–123 (2019)

    Article  Google Scholar 

  34. Deng, J., Guo, J., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. arXiv preprint arXiv:1801.07698 (2018)

  35. Yu, J. et al.: Deep metric learning with dynamic margin hard sampling loss for face verification. Signal Image Video Process. 1–8 (2019)

  36. Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: Computer Vision and Pattern Recognition IEEE, pp 3476–3483 (2013)

Download references

Author information

Authors and Affiliations

  1. School of Communication and Information Engineering, Shanghai University, Shanghai, China

    Zhongkui Fan & Ye-peng Guan

  2. Key Laboratory of Advanced Displays and System Application, Ministry of Education, Shanghai, China

    Ye-peng Guan

Authors
  1. Zhongkui Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ye-peng Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ye-peng Guan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, Z., Guan, Yp. A deep learning framework for face verification without alignment. J Real-Time Image Proc 18, 999–1009 (2021). https://doi.org/10.1007/s11554-020-01037-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01037-z

Keywords

Search

Navigation