Skip to main content
SpringerLink
Log in

Cancelable multi-biometric recognition system based on deep learning

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we propose a cancelable multi-biometric face recognition method that uses multiple convolutional neural networks (CNNs) to extract deep features from different facial regions. We also propose a new CNN architecture that exploits batch normalization, depth concatenation and a residual learning framework. The proposed method adopts a region-based technique in which face, eyes, nose and mouth regions are detected from the original face images. Multiple CNNs are used to extract deep features from each region, and then, a fusion network combines these features. Moreover, to provide user’s privacy and increase the system resistance against spoof attacks, a cancelable biometric technique using bio-convolving encryption is performed on the final facial descriptor. Our experiments on the FERET, LFW and PaSC datasets show excellent and competitive results compared to state-of-the-art methods in terms of recognition accuracy, specificity, precision, recall and fscore.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Choi, J.Y.: Spatial pyramid face feature representation and weighted dissimilarity matching for improved face recognition. Vis. Comput. 34, 1535–1549 (2018)

    Article  Google Scholar 

  2. Yongjie, Ch., Lindu, Z., Touqeer, A.: Multiple feature subspaces analysis for single sample per person face recognition. The visual figure 7: encryption and decryption operations on a number of facial images using different de-convolution masks. Computer 35(2), 239–256 (2019)

    Google Scholar 

  3. Li, K., Yi, J., Muhammad, W., Ruize, H., Jiongwei, Ch.: Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01627-4

  4. Cheng, Y., Jiao, L., Cao, X., Li, Z.: Illumination-insensitive features for face recognition. Vis. Comput. 33, 1483–1493 (2017)

    Article  Google Scholar 

  5. Yueming, W., Gang, P., Jianzhuang, L.: A deformation model to reduce the effect of expressions in 3D face recognition. Vis. Comput. 27(5), 333–345 (2011)

    Article  Google Scholar 

  6. Yujie, L., Deng, Y., Xiaoming, Ch., Zongmin, L., Jianping, F.: TOP-SIFT: the selected SIFT descriptor based on dictionary learning. Vis. Comput. 35, 667–677 (2019)

    Article  Google Scholar 

  7. Leila, K., Mehrez, A., Ali, D.: Image classification by combining local and global features. Vis. Comput. 35(5), 679–693 (2019)

    Article  Google Scholar 

  8. Baozhen, L., Hang, W., Weihua, S., Wenchang, Z., Jinggong, S.: Rotation-invariant object detection using Sector-ring HOG and boosted random ferns. Vis. Comput. 34(5), 707–719 (2018)

    Article  Google Scholar 

  9. Fengping, A., Zhiwen, L.: Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01635-4

  10. Jianfeng, Z., Xia, M., Jian, Z.: Learning deep facial expression features from image and optical flow sequences using 3D CNN. Vis. Comput. 34(10), 1461–1475 (2018)

    Article  Google Scholar 

  11. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

  12. Gaurav, T., Kuldeep, S., Dinesh, V.: Convolutional neural networks for crowd behaviour analysis: a survey. Vis. Comput. 35(5), 753–776 (2019)

    Article  Google Scholar 

  13. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: CVPR, pp. 815–823 (2015)

  14. Heekyung, Y., Kyungha, M.: Classification of basic artistic media based on a deep convolutional approach. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01627-4

    Article  Google Scholar 

  15. Wu, X., He, R., Sun, Z.,; Tan, T.: A light CNN for deep face representation with noisy labels. In: Proceedings of the IEEE Transactions on Information Forensics and Security, pp. 2884–2896 (2018)

  16. Deng, J., Guo, J., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. arXiv preprint arXiv:1801.07698 (2018)

  17. Liu, Y., Li, H., Wang, X.: Rethinking feature discrimination and polymerization for large-scale recognition. arXiv preprint arXiv:1710.00870 (2017)

  18. Zheng, Y., Pal, D., Savvides, M.: Ring loss: convex feature normalization for face recognition. In: CVPR (2018)

  19. Lobache, O.: Direct visualization of cryptographic keys for enhanced security. Vis. Comput. 34, 1749–1759 (2018)

    Article  Google Scholar 

  20. Ratha, N., Chikkerur, S., Connell, J., Bolle, R.: Generating cancelable fingerprint templates. IEEE Trans. Pattern Anal. Mach. Intell. 29, 561–572 (2007)

    Article  Google Scholar 

  21. Teoh, A., Goh, A., Ngo, D.: Random multispace quantization as an analytic mechanism for biohashing of biometric and random identity inputs. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1892–1901 (2006)

    Article  Google Scholar 

  22. Polash, P., Gavrilova, M., Klimenko, S.: Situation awareness of cancelable biometric system. Vis. Comput. 30, 1059–1067 (2014)

    Article  Google Scholar 

  23. Klare, B., Klein, B., Jain, A.: Pushing the frontiers of unconstrained face detection and recognition: Iarpa janus benchmark a. In: Proceedings of the IEEE CVPR, pp. 1931–1939 (2015)

  24. Je-Kang, P., Dong, K.: Unified convolutional neural network for direct facial keypoints detection. Vis. Comput. (2018). https://doi.org/10.1007/s00371-018-1561-3

    Article  Google Scholar 

  25. Li, L., Jun, Z., Fei, J., Li, S.: An incremental face recognition system based on deep learning. In: Proceedings of the Fifteenth IAPR International Conference on Machine Vision Applications (MVA) Nagoya University, Nagoya, Japan, pp. 238–241 (2017)

  26. Xujie, L., Hui, H., Hanli, Z., Yandan, W., Mingxiao, H.: Learning a convolutional neural network for propagation-based stereo image segmentation. Vis. Comput. (2018). https://doi.org/10.1007/s00371-018-1582-y

  27. Xiaoxiao, L., Qingyang, X., Ning, W.: A survey on deep neural network-based image captioning. Vis. Comput. 35(3), 445–470 (2019)

    Article  Google Scholar 

  28. Sun, Y., Wang, X., Tang, X.: Deeply learned face representations are sparse, selective, and robust. In: Proceedings of the IEEE CVPR, pp. 2892–2900 (2015)

  29. Sun, Y., Wang, X., Tang, X.: Sparsifying neural network connections for face recognition. In: Proceedings of the IEEE CVPR (2016)

  30. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE CVPR, pp. 1701–1708 (2014)

  31. Hao, G., Baozhong, C.: How do deep convolutional features affect tracking performance: an experimental study. Vis. Comput. 34(12), 1701–1711 (2018)

    Article  Google Scholar 

  32. Masi, I., Rawls, S., Natarajan, P.: Pose-aware face recognition in the wild. In: Proceedings of the IEEE CVPR, pp. 4838–4846 (2016)

  33. Taigman, Y., Yang, M., Wolf, L.: Web-scale training for face identification. In: Proceedings of the IEEE CVPR, pp. 2746–2754 (2015)

  34. Kurban, O., Yildirim, T., Bi̇lgi̇ç, A.: A multi-biometric recognition system based on deep features of face and gesture energy image. In: Proceedings of the IEEE International Conference on Innovations in Intelligent Systems and Applications (INISTA), pp. 361 – 364 (2017)

  35. Li, L., Jun, Z., Fei, J., Li, S.: An incremental face recognition system based on deep learning. In: Proceedings of the Fifteenth IAPR International Conference on Machine Vision Applications (MVA), pp. 238–241 (2017)

  36. Feng, Y.C., Yuen, P.C., Jain, A.K.: A hybrid approach for generating secure and discriminating face template. IEEE Trans. Inf. Forensics Secur. 5(1), 103–117 (2010)

    Article  Google Scholar 

  37. Yang, W., Wang, S., Zheng, G., Chaudhry, J., Valli, C.: ECB4CI: an enhanced cancelable biometric system for securing critical infrastructures. J. Supercomput. 74, 4893–4909 (2018)

    Article  Google Scholar 

  38. Paul, P.P., Gavrilova, M.: A novel cross folding algorithm for multimodal cancelable biometrics. Int. J. Softw. Sci. Comput. Intell. 4(3), 21–38 (2012)

    Article  Google Scholar 

  39. Alder, A.: Sample images can be independently restored from face recognition templates. Electr. Comput. Eng. 2, 1163–1166 (2003)

    Google Scholar 

  40. Bedad, F., Adjoudj, R.: Multi-biometric template protection: an overview. In: Proceedings of International Conference in Artificial Intelligence in Renewable Energetic Systems, pp. 70–79 (2018)

  41. Natgunanathan, I., Mehmood, A., Xiang, Y., Hua, G., Li, G., Bangay, S.: An overview of protection of privacy in multibiometrics. Multimed. Tools Appl. 77, 6753–6773 (2018)

    Article  Google Scholar 

  42. Droandi, G., Barni, M., Lazzeretti, R., Pignata, T.: SEMBA:SEcure multi-biometric authentication. arXiv:1803.10758 (2018)

  43. Teoh, A., Ngo, D., Goh, A.: Biohashing: two factor authentication featuring fingerprint data and tokenised random number. Pattern Recognit. 37(11), 2245–2255 (2004)

    Article  Google Scholar 

  44. Patel, V.M., Ratha, N.K., Chellappa, R.: Cancelable biometrics: a review. IEEE Signal Process. Mag. 32(5), 54–65 (2015)

    Article  Google Scholar 

  45. Marco, C., Oscar, D., Cayetano, G., Mario, H.: ENCARA2: real-time detection of multiple faces at different resolutions in video streams. J. Vis. Commun. Image Represent. 18(2), 130–140 (2007)

    Article  Google Scholar 

  46. Yuan, Z., Liu, Y., Yue, J., Li, J., Yang, H.: CORAL: coarse-grained reconfigurable architecture for convolutional neural networks. In: Proceedings of the IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), pp. 1–6 (2017)

  47. Ioffe, S., Szegedy, Ch.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv: 1502.03167v3 [cs.LG], pp. 1–11 (2015)

  48. Zhang, X., Ren, Sh., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  49. Liu, W., Jia, Y., Sermanet, P., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  50. Xiaoliang, Z., Zijian, C.: Dual-modality spatiotemporal feature learning for spontaneous facial expression recognition in e-learning using hybrid deep neural network. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01660-3

    Article  Google Scholar 

  51. Dandan, L., Huagang, L., Zhenbo, Y., Yipu, Z.: Deep convolutional BiLSTM fusion network for facial expression recognition. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01636-3

    Article  Google Scholar 

  52. Hasnat, A., Bohn´e, J., Milgram, J., Gentric, S., Chen, L.: DeepVisage: making face recognition simple yet with powerful generalization skills. In: Proceedings of the CVPR, pp. 1–12 (2017)

  53. https://www.nist.gov/itl/iad/image-group/color-feret-database

  54. Huang, G., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst (2007)

  55. Beveridge, J., Phillips, J., Bolme, D., Draper, B., Givens, G., Lui, Y., Teli, M., Zhang, H., Scruggs, W., Bowyer, K.: The challenge of face recognition from digital point-and-shoot cameras. In: Proceedings of IEEE International Conference on Biometrics: Theory, Applications Systems, pp. 1–8 (2013)

  56. Wu, X., He, R., Sun, Z., Tan, T.: A Light CNN for deep face representation with noisy labels. In: CVPR, pp. 1–13 (2017)

  57. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: SphereFace: deep hypersphere embedding for face recognition. In: CVPR (2017)

  58. Liu, J., Deng, Y., Huang, C.: Targeting ultimate accuracy: face recognition via deep embedding. arXiv:1506.07310 (2015)

  59. Sun, Y., Liang, D., Wang, X., Tang, X.: DeepID3: face recognition with very deep neural networks. arXiv:1502.00873 (2015)

  60. Ding, C., Tao, D.: Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1002–1014 (2017)

    Article  MathSciNet  Google Scholar 

  61. Shu, X., Liyan, Z., Jinhui, T., Guo-Sen, X., Shuicheng, Y.: Computational face reader. In: MMM, Part I LNCS 9516, pp. 114–126 (2016)

  62. Shu, X., Jinhui, T., Zechao, L., Hanjiang, L., Liyan, Z., Shuicheng, Y.: Personalized age progression with bi-level aging dictionary learning. IEEE Trans. Pattern Anal. Mach. Intell. (2017)

  63. Shu, X., Jinhui, T., Hanjiang, L., Zhiheng, N., Shuicheng, Y.: Kinship-guided age progression. Pattern Recognit. 59, 156–167 (2016). https://doi.org/10.1016/j.patcog.2015.12.015

    Article  Google Scholar 

  64. Lin, Y., Xun, L.: A Cancelable multi-biometric template generation algorithm based on bloom filter. In: International Conference on Algorithms and Architectures for Parallel Processing, pp. 547–559 (2018)

Download references

Author information

Authors and Affiliations

  1. Electronics and Communication Department, Delta Academy for Engineering, Mansoura, Egypt

    Essam Abdellatef

  2. Department of Computer Science and Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Nabil A. Ismail & Salah Eldin S. E. Abd Elrahman

  3. Department of Computer Science, Durham University, Durham, DH1 3LE, UK

    Khalid N. Ismail

  4. Information Technology Department, Faculty of Computers and Information, Menoufia University, Menoufia, Egypt

    Khalid N. Ismail

  5. Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Mohamed Rihan & Fathi E. Abd El-Samie

Authors
  1. Essam Abdellatef
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nabil A. Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salah Eldin S. E. Abd Elrahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khalid N. Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohamed Rihan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fathi E. Abd El-Samie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Essam Abdellatef.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abdellatef, E., Ismail, N.A., Abd Elrahman, S.E.S.E. et al. Cancelable multi-biometric recognition system based on deep learning. Vis Comput 36, 1097–1109 (2020). https://doi.org/10.1007/s00371-019-01715-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01715-5

Keywords

Search

Navigation