Skip to main content
SpringerLink
Log in

Prominent facial feature and hybrid learning method-based advanced face detector robust to head-up, head-down, and arbitrary rotation cases

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Face detection is important for computer vision. Until now, the Viola and Jones algorithm is still the most popular face detector embedded in digital cameras. Although it has good performance in frontal face detection, it may not perform well in detecting rotated, head-up, and head-down faces. In this paper, we propose a more robust face detector based on prominent feature extraction. We find that, in addition to using eyes and mouth, which have been adopted by other algorithms, noses and ears are also important clues for face detection. Particularly, for a profile face, only a single eye can be detected. For head-up or head-down faces, eyes and mouth may not be well detected. However, in these cases, noses and ears can still be detected well and we can apply these features to improve the accuracy of rotated, head-up, and head-down face detection. To well detect these prominent features, in addition to the Viola–Jones detector, we also apply edge and color information and use the refocus algorithm based on the convolutional neural network. Simulations conducted on several popular multi-view face databases show that the proposed face detector can attain higher detection rates and is robust to rotated, head-up, and head-down cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  2. Viola, P., Jones, M.J.: Rapid object detection using a boosted cascade of simple features. In: IEEE Conference Computer Vision and Pattern Recognition, pp. 511–518 (2001)

  3. Yong, Y., Ding, X.: Real-time multi-view face detection and pose estimation based on cost-sensitive Adaboost. Tsing Hua Science and Technology 10(2), 152–157 (2005)

    Article  Google Scholar 

  4. Huang, C., Ai, H., Wu, B., Lao, S.: Boosting nested cascade detector for multi-view face detection. In: International Conference Pattern Recognition, pp. 415–418 (2004)

  5. Kang, S., Choi, B., Jo, D.: Faces detection method based on skin color modeling. J. Syst. Architect. 64, 100–109 (2016)

    Article  Google Scholar 

  6. Ghimire, D., Lee, J.: A robust face detection method based on skin color and edges. J. Inf. Process. Syst. 9(1), 141–156 (2013)

    Article  Google Scholar 

  7. Hsu, R.L., Abdel-Mottaleb, M., Jain, A.K.: Face detection in color images. IEEE Trans. Patt. Anal. Mach. Intell. 24(5), 696–706 (2002)

    Article  Google Scholar 

  8. Kaddouhi, S.E., Saaidi, A., Abarkan, M.: A new robust face detection method based on corner points, In. J. Softw. Eng. Appl. 8(11), 25–40 (2014)

    Google Scholar 

  9. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Conference Computer Vision and Pattern Recognition, pp. 2879–2886 (2012)

  10. Cevikalp, H., Triggs, B.: Efficient object detection using cascades of nearest convex model classifiers. In: IEEE Conference Computer Vision and Pattern Recognition, pp. 3138–3145 (2012)

  11. Ghiasi G., Fowlkes, C.: Occlusion coherence: Localizing occluded faces with a hierarchical deformable part model. In: IEEE Conference Computer Vision and Pattern Recognition, pp. 2385–2392 (2014)

  12. Tsai, Y.H., Lee, Y.C., Ding, J.J., Chang, R.Y., Hsu, M.C.: Robust in-plane and out-of-plane face detection algorithm using frontal face detector and symmetry extension. Image Vis. Comput. 78, 26–41 (2018)

    Article  Google Scholar 

  13. Li, H., Lin, Z., Shen, X., Brandt, J., Hua, G.: A convolutional neural network cascade for face detection. In: International Conference Computer Vision, pp. 5325–5334 (2015)

  14. Farfade, S.S., Saberian, M., Li, L.J.: Multi-view face detection using deep convolutional neural networks. In: ACM International Conference Multimedia Retrieval, pp. 643–650 (2015)

  15. Zhang, C., Zhang, Z.: Improving multiview face detection with multi-task deep convolutional neural networks. In: IEEE Winter Conference Applications of Computer Vision, pp. 1036–1041 (2014)

  16. Jourabloo, A., Liu, X.: Large-pose face alignment via CNN-based dense 3D model fitting. In: IEEE Conference Computer Vision and Pattern Recognition, pp. 4188–4196 (2016)

  17. Alotaibi, A., Mahmood, A.: Deep face liveness detection based on nonlinear diffusion using convolution neural network. SIViP 11(4), 713–720 (2017)

    Article  Google Scholar 

  18. Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: European Conference Computer Vision, pp. 94–108 (2014)

  19. Yang, B., Yan, J., Lei, Z., Li, S.Z.: Aggregate channel features for multi-view face detection. In: IEEE International Joint Conference Biometrics, pp. 1–8 (2014)

  20. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  21. Xu, L., Ren, J., Yan, Q., Liao, R., Jia, J.: Deep edge-aware filters. In: International Conference Machine Learning, pp. 1669–1678 (2015)

  22. Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Trans. Intel. Syst. Tech. 2(3), 27 (2011)

    Article  Google Scholar 

  23. Hu, J.L., Deng, J.B., Zou, S.S.: A novel algorithm for color space conversion model from CMYK to LAB. J. Multimedia 5(2), 159–166 (2010)

    Article  Google Scholar 

  24. Cheng, H.D., Shi, X.J.: A simple and effective histogram equalization approach to image enhancement. Dig. Signal Proc. 14(2), 158–170 (2004)

    Article  Google Scholar 

  25. The FEI database will be available at http://fei.edu.br/~cet/facedatabase.html. Accessed Mar 2006

  26. Liu, J.Z., Li, W.Q., Tian, Y.P.: Automatic thresholding of gray-level pictures using two-dimension Otsu method. In: IEEE International Conference Circuits and Systems. pp. 325–327 (1991)

  27. Chen, C.Y., Ding, J.J., Huang, C.W., Advanced ear detection algorithm using faster R-CNN, refocus filters, and the gradient map. In: IEEE International Conference Digital Signal Processing, pp. 1–5 (2018)

  28. Chen, C.Y., Ding, J.J., Hsu, H.W., Lee, Y.C., Advanced orientation robust face detection algorithm using prominent features and hybrid learning techniques. In: IEEE Conference Visual Communications and Image Processing, pp. 1–4 (2018)

  29. Xu, L., Lu, C., Xu, Y., Jia, J.: Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30(6), 174 (2011)

    Google Scholar 

  30. The Pointing’04. http://www-prima.inrialpes.fr/FGnet/data/13-MoeslundHead/Pointing04/. Accessed Aug 2004

  31. The USTB. http://www1.ustb.edu.cn/resb/en/visit/visit.htm. Accessed Aug 2002

  32. The CVL. http://www.lrv.fri.uni-lj.si/facedb.html. Accessed 1999

  33. The GTAV. https://gtav.upc.edu/en/research-areas/face-database. Accessed 2011

  34. The Color Feret. https://www.nist.gov/itl/iad/image-group/color-feret-database. Accessed Jan 2011

  35. Liao, S., Jain, A.K., Li, S.Z.: A fast and accurate unconstrained face detector. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 211–223 (2016)

    Article  Google Scholar 

  36. Orozco, J., Martinez, B., Pantic, M.: Empirical analysis of cascade deformable models for multi-view face detection. Image Vis. Comput. 42, 47–61 (2015)

    Article  Google Scholar 

  37. Chen, J., Deng, Y., Bai, G., Su, G.: Face image quality assessment based on learning to rank. IEEE Signal Process. Lett. 22(1), 90–94 (2014)

    Article  Google Scholar 

  38. Najibi, M., Samangouei, P., Chellappa, R., Davis, L.: SSH: Single stage headless dace detector. In: International Conference Computer Vision, pp. 4875–4884 (2017)

Download references

Author information

Authors and Affiliations

  1. Graduate Institute of Communication Engineering, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd., Taipei, 10617, Taiwan

    Chien-Yu Chen, Jian-Jiun Ding & Hsuan-Wei Hsu

Authors
  1. Chien-Yu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian-Jiun Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hsuan-Wei Hsu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian-Jiun Ding.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, CY., Ding, JJ. & Hsu, HW. Prominent facial feature and hybrid learning method-based advanced face detector robust to head-up, head-down, and arbitrary rotation cases. SIViP 15, 147–154 (2021). https://doi.org/10.1007/s11760-020-01729-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01729-w

Keywords

Search

Navigation