Skip to main content
SpringerLink
Log in

A Survey of Deep Facial Attribute Analysis

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Facial attribute analysis has received considerable attention when deep learning techniques made remarkable breakthroughs in this field over the past few years. Deep learning based facial attribute analysis consists of two basic sub-issues: facial attribute estimation (FAE), which recognizes whether facial attributes are present in given images, and facial attribute manipulation (FAM), which synthesizes or removes desired facial attributes. In this paper, we provide a comprehensive survey of deep facial attribute analysis from the perspectives of both estimation and manipulation. First, we summarize a general pipeline that deep facial attribute analysis follows, which comprises two stages: data preprocessing and model construction. Additionally, we introduce the underlying theories of this two-stage pipeline for both FAE and FAM. Second, the datasets and performance metrics commonly used in facial attribute analysis are presented. Third, we create a taxonomy of state-of-the-art methods and review deep FAE and FAM algorithms in detail. Furthermore, several additional facial attribute related issues are introduced, as well as relevant real-world applications. Finally, we discuss possible challenges and promising future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Amazon Mechanical Turk. https://www.mturk.com/.

References

  • Belghazi, M. I., Rajeswar, S., Mastropietro, O., Rostamzadeh, N., Mitrovic, J., & Courville, A. (2018). Hierarchical adversarially learned inference. arXiv preprint arXiv:1802.01071.

  • Berg, T., & Belhumeur, P. N. (2013). Poof: Part-based one-vs.-one features for fine-grained categorization, face verification, and attribute estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 955–962). IEEE.

  • Berthelot, D., Raffel, C., Roy, A., & Goodfellow, I. (2019). Understanding and improving interpolation in autoencoders via an adversarial regularizer. In Proceedings of the international conference on learning representations (ICLR).

  • Bourdev, L., Maji, S., & Malik, J. (2011). Describing people: A poselet-based approach to attribute classification. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 1543–1550). IEEE.

  • Bourdev, L., & Malik, J. (2009). Poselets: Body part detectors trained using 3d human pose annotations. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 1365–1372). IEEE.

  • Cao, C., Lu, F., Li, C., Lin, S., & Shen, X. (2019a). Makeup removal via bidirectional tunable de-makeup network. IEEE Transactions on Multimedia, 21, 2750–2761.

    Article  Google Scholar 

  • Cao, J., Hu, Y., Yu, B., He, R., & Sun, Z. (2019b). 3D aided duet gans for multi-view face image synthesis. IEEE Transactions on Information Forensics and Security (TIFS), 14(8), 2028–2042.

    Article  Google Scholar 

  • Cao, J., Li, Y., & Zhang, Z. (2018). Partially shared multi-task convolutional neural network with local constraint for face attribute learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 4290–4299).

  • Chang, H., Lu, J., Yu, F., & Finkelstein, A. (2018). Pairedcyclegan: Asymmetric style transfer for applying and removing makeup. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 40–48).

  • Chen, J. C., Ranjan, R., Sankaranarayanan, S., Kumar, A., Chen, C. H., Patel, V. M., et al. (2018). Unconstrained still/video-based face verification with deep convolutional neural networks. International Journal of Computer Vision (IJCV), 126(2–4), 272–291.

    Article  MathSciNet  Google Scholar 

  • Chen, L., Zhang, Q., & Li, B. (2014). Predicting multiple attributes via relative multi-task learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1027–1034).

  • Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016). Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in neural information processing systems (NIPS) (pp. 2172–2180).

  • Chen, Y. C., Shen, X., Lin, Z., Lu, X., Pao, I., Jia, J., et al. (2019). Semantic component decomposition for face attribute manipulation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 9859–9867).

  • Chhabra, S., Singh, R., Vatsa, M., & Gupta, G. (2018) Anonymizing k-facial attributes via adversarial perturbations. In Proceedings of the international joint conference on artificial intelligence (IJCAI) (pp. 656–662).

  • Choi, Y., Choi, M., & Kim, M. (2018). Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 8789–8797).

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.

    MATH  Google Scholar 

  • Ding, H., Zhou, H., Zhou, S. K., & Chellappa, R. (2018). A deep cascade network for unaligned face attribute classification. In Proceedings of the conference on artificial intelligence (AAAI).

  • Dong, Q., Gong, S., & Zhu, X. (2017). Class rectification hard mining for imbalanced deep learning. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 1869–1878). IEEE.

  • Dorta, G., Vicente, S., Campbell, N. D., & Simpson, I. (2018). The GAN that warped: Semantic attribute editing with unpaired data. arXiv preprint arXiv:1811.12784.

  • Egger, B., Schönborn, S., Schneider, A., Kortylewski, A., Morel-Forster, A., Blumer, C., et al. (2018). Occlusion-aware 3d morphable models and an illumination prior for face image analysis. International Journal of Computer Vision (IJCV), 126, 1269–1287.

    Article  Google Scholar 

  • Fan, Q., Gabbur, P., & Pankanti, S. (2013). Relative attributes for large-scale abandoned object detection. In Proceedings of the IEEE International conference on computer vision (ICCV) (pp. 2736–2743).

  • Fang, Y., & Yuan, Q. (2018). Attribute-enhanced metric learning for face retrieval. EURASIP Journal on Image and Video Processing, 2018(1), 44.

    Article  Google Scholar 

  • Fathy, M. E., Patel, V. M., & Chellappa, R. (2015). Face-based active authentication on mobile devices. In Proceedings of the IEEE conference on acoustics, speech and signal processing (ICASSP) (pp. 1687–1691). IEEE.

  • Fukui, H., Hirakawa, T., Yamashita, T., & Fujiyoshi, H. (2019). Attention branch network: Learning of attention mechanism for visual explanation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 10705–10714).

  • Gkioxari, G., Girshick, R., & Malik, J. (2015). Actions and attributes from wholes and parts. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 2470–2478). IEEE.

  • Gonzalez-Garcia, A., Modolo, D., & Ferrari, V. (2018). Do semantic parts emerge in convolutional neural networks? International Journal of Computer Vision (IJCV), 126(5), 476–494.

    Article  MathSciNet  Google Scholar 

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. (2014). Generative adversarial nets. In Advances in neural information processing systems (NIPS) (pp. 2672–2680).

  • Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. In Proceedings of the international conference on learning representations (ICLR).

  • Günther, M., Costa-Pazo, A., Ding, C., Boutellaa, E., Chiachia, G., Zhang, H., et al. (2013). The 2013 face recognition evaluation in mobile environment. In Proceedings of the international conference on biometrics (ICB) (pp. 1–7). IEEE.

  • Günther, M., Rozsa, A., & Boult, T. E. (2017). AFFACT: Alignment-free facial attribute classification technique. In Proceedings of the IEEE international joint conference on biometrics (IJCB) (pp. 90–99). IEEE.

  • Hadid, A., Heikkila, J., Silvén, O., & Pietikainen, M. (2007). Face and eye detection for person authentication in mobile phones. In Proceedings of the ACM/IEEE international conference on distributed smart cameras (pp. 101–108). IEEE.

  • Haixiang, G., Yijing, L., Shang, J., Mingyun, G., Yuanyue, H., & Bing, G. (2017). Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 73, 220–239.

    Article  Google Scholar 

  • Han, H., Jain, A. K., Shan, S., & Chen, X. (2017). Heterogeneous face attribute estimation: A deep multi-task learning approach. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 40, 2597–2609.

    Article  Google Scholar 

  • Hand, E. M., Castillo, C. D., & Chellappa, R. (2018a). Doing the best we can with what we have: Multi-label balancing with selective learning for attribute prediction. In Proceedings of the conference on artificial intelligence (AAAI) (pp. 6878–6885).

  • Hand, E. M., Castillo, C. D., & Chellappa, R. (2018b). Predicting facial attributes in video using temporal coherence and motion-attention. In Proceedings of the IEEE winter conference on applications of computer vision (WACV) (pp. 84–92). IEEE.

  • Hand, E. M., & Chellappa, R. (2017). Attributes for improved attributes: A multi-task network utilizing implicit and explicit relationships for facial attribute classification. In Proceedings of the conference on artificial intelligence (AAAI) (pp. 4068–4074).

  • He, D., Xia, Y., Qin, T., Wang, L., Yu, N., Liu, T., & Ma, W. Y. (2016a). Dual learning for machine translation. In Advances in neural information processing systems (NIPS) (pp. 820–828).

  • He, K., Fu, Y., & Xue, X. (2017). A jointly learned deep architecture for facial attribute analysis and face detection in the wild. arXiv preprint arXiv:1707.08705.

  • He, K., Fu, Y., Zhang, W., Wang, C., Jiang, Y. G., Huang, F., & Xue, X. (2018a). Harnessing synthesized abstraction images to improve facial attribute recognition. In Proceedings of the international joint conference on artificial intelligence (IJCAI).

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016b). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 770–778).

  • He, R., Tan, T., Davis, L., & Sun, Z. (2018b). Learning structured ordinal measures for video based face recognition. Pattern Recognition, 75, 4–14.

    Article  Google Scholar 

  • He, R., Wu, X., Sun, Z., & Tan, T. (2018c). Wasserstein CNN: Learning invariant features for NIR-VIS face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41, 1761–1773.

    Article  Google Scholar 

  • He, Z., Zuo, W., Kan, M., Shan, S., & Chen, X. (2019). Attgan: Facial attribute editing by only changing what you want. IEEE Transactions on Image Processing (TIP), 28, 5464–5478.

    Article  MathSciNet  MATH  Google Scholar 

  • Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in neural information processing systems (NIPS) (pp. 6626–6637).

  • Hu, Y., Wu, X., Yu, B., He, R., & Sun, Z. (2018). Pose-guided photorealistic face rotation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Huang, C., Li, Y., Change Loy, C., & Tang, X. (2016). Learning deep representation for imbalanced classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 5375–5384).

  • Huang, C., Li, Y., Chen, C. L., & Tang, X. (2019). Deep imbalanced learning for face recognition and attribute prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). https://doi.org/10.1109/TPAMI.2019.2914680.

  • Huang, G. B., Mattar, M., Berg, T., & Learned-Miller, E. (2008). Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in ‘Real-Life’ images: Detection, alignment, and recognition.

  • Huang, H., He, R., Sun, Z., Tan, T., et al. (2018a). Introvae: Introspective variational autoencoders for photographic image synthesis. In Advances in neural information processing systems (NIPS) (pp. 52–63).

  • Huang, H., Song, L., He, R., Sun, Z., & Tan, T. (2018b). Variational capsules for image analysis and synthesis. arXiv preprint arXiv:1807.04099.

  • Kalayeh, M. M., Gong, B., & Shah, M. (2017). Improving facial attribute prediction using semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 4227–4235). IEEE.

  • Kazemi, V., & Sullivan, J. (2014). One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1867–1874).

  • Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. In Proceedings of the international conference on learning representations (ICLR).

  • Kumar, N., Belhumeur, P., & Nayar, S. (2008). Facetracer: A search engine for large collections of images with faces. In Proceedings of the European conference on computer vision (ECCV) (pp. 340–353). Springer.

  • Kumar, N., Berg, A., Belhumeur, P. N., & Nayar, S. (2011). Describable visual attributes for face verification and image search. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 33(10), 1962–1977.

    Article  Google Scholar 

  • Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009). Attribute and simile classifiers for face verification. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 365–372). IEEE.

  • Lampert, C. H., Nickisch, H., & Harmeling, S. (2009). Learning to detect unseen object classes by between-class attribute transfer. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 951–958). IEEE.

  • Lample, G., Zeghidour, N., Usunier, N., Bordes, A., Denoyer, L., et al. (2017). Fader networks: Manipulating images by sliding attributes. In Advances in neural information processing systems (NIPS) (pp. 5967–5976).

  • Larsen, A. B. L., Sønderby, S. K., Larochelle, H., & Winther, O. (2016). Autoencoding beyond pixels using a learned similarity metric. In Proceedings of the IEEE international conference on machine learning (ICML) (pp. 1558–1566).

  • Le, V., Brandt, J., Lin, Z., Bourdev, L., & Huang, T. S. (2012). Interactive facial feature localization. In Proceedings of the European conference on computer vision (ECCV) (pp. 679–692). Springer.

  • Li, H. Y., Dong, W. M., & Hu, B. G. (2018a). Facial image attributes transformation via conditional recycle generative adversarial networks. Journal of Computer Science and Technology (JCST), 33(3), 511–521.

    Article  Google Scholar 

  • Li, J., Zhao, F., Feng, J., Roy, S., Yan, S., & Sim, T. (2018b). Landmark free face attribute prediction. IEEE Transactions on Image Processing (TIP), 27(9), 4651–4662.

    Article  MathSciNet  Google Scholar 

  • Li, M., Zuo, W., & Zhang, D. (2016). Deep identity-aware transfer of facial attributes. arXiv preprint arXiv:1610.05586.

  • Li, T., Qian, R., Dong, C., Liu, S., Yan, Q., Zhu, W., & Lin, L. (2018c). Beautygan: Instance-level facial makeup transfer with deep generative adversarial network. In Proceedings of the ACM multimedia conference on multimedia conference (ACMMM) (pp. 645–653). ACM.

  • Li, Y., Wang, R., Liu, H., Jiang, H., Shan, S., & Chen, X. (2015). Two birds, one stone: Jointly learning binary code for large-scale face image retrieval and attributes prediction. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 3819–3827). IEEE.

  • Liu, A. H., Liu, Y. C., Yeh, Y. Y., & Wang, Y. C. F. (2018). A unified feature disentangler for multi-domain image translation and manipulation. In Advances in neural information processing systems (NIPS) (pp. 2591–2600).

  • Liu, M. Y., Breuel, T., & Kautz, J. (2017). Unsupervised image-to-image translation networks. In Advances in neural information processing systems (NIPS) (pp. 700–708).

  • Liu, Y., Li, Q., & Sun, Z. (2019). Attribute-aware face aging with wavelet-based generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 11877–11886).

  • Liu, Z., Luo, P., Wang, X., & Tang, X. (2015). Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 3730–3738).

  • Lu, Y., Kumar, A., Zhai, S., Cheng, Y., Javidi, T., & Feris, R. (2017). Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (Vol. 1, p. 6).

  • Lu, Y., Tai, Y. W., & Tang, C. K. (2018a) Attribute-guided face generation using conditional cyclegan. In Proceedings of the European conference on computer vision (ECCV) (pp. 293–308). Springer.

  • Lu, Z., Hu, T., Song, L., Zhang, Z., & He, R. (2018b). Conditional expression synthesis with face parsing transformation. In Proceedings of the ACM international conference on multimedia (ACMMM) (pp. 1083–1091). ACM.

  • Luo, P., Wang, X., & Tang, X. (2013). A deep sum-product architecture for robust facial attributes analysis. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 2864–2871). IEEE.

  • Ma, L., Jia, X., Georgoulis, S., Tuytelaars, T., & Van Gool, L. (2018). Exemplar guided unsupervised image-to-image translation with semantic consistency. In Proceedings of the international conference on learning representations (ICLR).

  • Mahbub, U., Sarkar, S., & Chellappa, R. (2018). Segment-based methods for facial attribute detection from partial faces. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2018.2820048.

  • Meng, Z., Adluru, N., Kim, H. J., Fung, G., & Singh, V. (2018). Efficient relative attribute learning using graph neural networks. In Proceedings of the European conference on computer vision (ECCV) (pp. 552–567).

  • Miller, T. L., Berg, A. C., Edwards, J. A., Maire, M. R., White, R. M., Teh, Y. W., et al. (2007). Names and faces.

  • Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784.

  • Nguyen, H. M., Ly, N. Q., & Phung, T. T. (2018). Large-scale face image retrieval system at attribute level based on facial attribute ontology and deep neuron network. In Asian conference on intelligent information and database systems (pp. 539–549). Springer.

  • Nhan Duong, C., Luu, K., Gia Quach, K., Nguyen, N., Patterson, E., Bui, T. D., Le, N. (2019). Automatic face aging in videos via deep reinforcement learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 10013–10022).

  • Parikh, D., & Grauman, K. (2011). Relative attributes. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 503–510). IEEE.

  • Parkhi, O. M., Vedaldi, A., Zisserman, A., et al. (2015). Deep face recognition. In Proceedings of the British machine vision conference 2015, (BMVC) (Vol. 1, p. 6).

  • Perarnau, G., van de Weijer, J., Raducanu, B., & Álvarez, J. M. (2016). Invertible conditional gans for image editing. In Advances in neural information processing systems workshop on adversarial training (NIPSW).

  • Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2007). Object retrieval with large vocabularies and fast spatial matching. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1–8). IEEE.

  • Ranjan, R., Patel, V. M., & Chellappa, R. (2017). Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41, 121–135.

    Article  Google Scholar 

  • Ranjan, R., Sankaranarayanan, S., Castillo, C. D., & Chellappa, R. (2017). An all-in-one convolutional neural network for face analysis. In Proceedings of the IEEE international conference on automatic face & gesture recognition (FG) (pp. 17–24). IEEE.

  • Rao, Y., Lu, J., & Zhou, J. (2018). Learning discriminative aggregation network for video-based face recognition and person re-identification. International Journal of Computer Vision (IJCV), 127, 701–718.

    Article  Google Scholar 

  • Rozsa, A., Günther, M., Rudd, E. M., & Boult, T. E. (2016). Are facial attributes adversarially robust? In Processing of international conference on pattern recognition (ICPR) (pp. 3121–3127). IEEE.

  • Rozsa, A., Günther, M., Rudd, E. M., & Boult, T. E. (2017). Facial attributes: Accuracy and adversarial robustness. Pattern Recognition Letters, 124, 100–108.

    Article  Google Scholar 

  • Rudd, E. M., Günther, M., & Boult, T. E. (2016). Moon: A mixed objective optimization network for the recognition of facial attributes. In Proceedings of the European conference on computer vision (ECCV) (pp. 19–35). Springer.

  • Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2013). A semi-automatic methodology for facial landmark annotation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW) (pp. 896–903).

  • Saito, M., Matsumoto, E., & Saito, S. (2017). Temporal generative adversarial nets with singular value clipping. In Proceedings of the IEEE international conference on computer vision (pp. 2830–2839).

  • Samangouei, P., Patel, V. M., & Chellappa, R. (2017). Facial attributes for active authentication on mobile devices. Image and Vision Computing, 58, 181–192.

    Article  Google Scholar 

  • Sandeep, R. N., Verma, Y., & Jawahar, C. (2014). Relative parts: Distinctive parts for learning relative attributes. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3614–3621).

  • Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 815–823).

  • Sethi, A., Singh, M., Singh, R., & Vatsa, M. (2018). Residual codean autoencoder for facial attribute analysis. Pattern Recognition Letters, 119, 157–165.

    Article  Google Scholar 

  • Shen, W., & Liu, R. (2017). Learning residual images for face attribute manipulation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1225–1233). IEEE.

  • Shi, H., & Tao, L. (2018). Fine-grained visual comparison based on relative attribute quadratic discriminant analysis. IEEE Transactions on Systems, Man, and Cybernetics: Systems. https://doi.org/10.1109/TSMC.2018.2800092.

  • Shi, Z., Hospedales, T. M., & Xiang, T. (2015). Transferring a semantic representation for person re-identification and search. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 4184–4193). IEEE.

  • Singh, K. K., & Lee, Y. J. (2016). End-to-end localization and ranking for relative attributes. In Proceedings of the European conference on computer vision (ECCV) (pp. 753–769). Springer.

  • Smith, B. M., Zhang, L., Brandt, J., Lin, Z., & Yang, J. (2013). Exemplar-based face parsing. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3484–3491). IEEE.

  • Song, F., Tan, X., & Chen, S. (2014). Exploiting relationship between attributes for improved face verification. Computer Vision and Image Understanding, 122, 143–154.

    Article  Google Scholar 

  • Song, L., Cao, J., Song, L., Hu, Y., & He, R. (2019). Geometry-aware face completion and editing. In Proceedings of the conference on artificial intelligence (AAAI).

  • Song, L., Lu, Z., He, R., Sun, Z., Tan, T. (2018a). Geometry guided adversarial facial expression synthesis. In Proceedings of the ACM international conference on multimedia (ACMMM) (pp. 627–635). ACM.

  • Song, L., Zhang, M., Wu, X., & He, R. (2018b). Adversarial discriminative heterogeneous face recognition. In Proceedings of the conference on artificial intelligence (AAAI).

  • Sun, R., Huang, C., Shi, J., & Ma, L. (2018c). Mask-aware photorealistic face attribute manipulation. arXiv preprint arXiv:1804.08882.

  • Sun, Y., Chen, Y., Wang, X., & Tang, X. (2014). Deep learning face representation by joint identification-verification. In Advances in neural information processing systems (NIPS) (pp. 1988–1996).

  • Suo, J., Zhu, S. C., Shan, S., & Chen, X. (2010). A compositional and dynamic model for face aging. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 32(3), 385–401.

    Article  Google Scholar 

  • Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the conference on artificial intelligence (AAAI) (Vol. 4, p. 12).

  • Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. In Proceedings of the international conference on learning representations (ICLR).

  • Taherkhani, F., Nasrabadi, N. M., & Dawson, J. (2018). A deep face identification network enhanced by facial attributes prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW) (pp. 553–560).

  • Toderici, G., O’malley, S.M., Passalis, G., Theoharis, T., Kakadiaris, I.A.: Ethnicity-and gender-based subject retrieval using 3-d face-recognition techniques. International Journal of Computer Vision (IJCV) 89(2-3), 382–391 (2010).

  • Trokielewicz, M., Czajka, A., & Maciejewicz, P. (2019). Iris recognition after death. IEEE Transactions on Information Forensics and Security, 14(6), 1501–1514.

    Article  Google Scholar 

  • Tropp, J. A., Gilbert, A. C., & Strauss, M. J. (2006). Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit. Signal Processing, 86(3), 572–588.

    MATH  Google Scholar 

  • Wang, J., Cheng, Y., & Schmidt Feris, R. (2016). Walk and learn: Facial attribute representation learning from egocentric video and contextual data. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2295–2304).

  • Wang, Y., Wang, S., Qi, G., Tang, J., & Li, B. (2018). Weakly supervised facial attribute manipulation via deep adversarial network. In Proceedings of the IEEE winter conference on applications of computer vision (WACV) (pp. 112–121). IEEE.

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing (TIP), 13(4), 600–612.

    Article  Google Scholar 

  • Wiles, O., Koepke, A., & Zisserman, A. (2018). Self-supervised learning of a facial attribute embedding from video. In Proceedings of the British machine vision conference (BMVC) (p. 302).

  • Wolf, L., Hassner, T., & Maoz, I. (2011). Face recognition in unconstrained videos with matched background similarity. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 529–534). IEEE Computer Society.

  • Wu, Y., & Ji, Q. (2017). Facial landmark detection: A literature survey. International Journal of Computer Vision (IJCV), 127, 115–142.

    Article  Google Scholar 

  • Xiao, F., & Jae Lee, Y. (2015) Discovering the spatial extent of relative attributes. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 1458–1466).

  • Xiao, T., Hong, J., & Ma, J. (2017). DNA-GAN: Learning disentangled representations from multi-attribute images. In Proceedings of the international conference on learning representations workshop track (ICLRW).

  • Xiao, T., Hong, J., & Ma, J. (2018). Elegant: Exchanging latent encodings with gan for transferring multiple face attributes. In Proceedings of the European conference on computer vision (ECCV) (pp. 168–184).

  • Yan, X., Yang, J., Sohn, K., & Lee, H. (2016). Attribute2image: Conditional image generation from visual attributes. In Proceedings of the European conference on computer vision (ECCV) (pp. 776–791). Springer.

  • Yue, Y., Finley, T., Radlinski, F., & Joachims, T. (2007). A support vector method for optimizing average precision. In Proceedings of the annual international ACM SIGIR conference on research and development in information retrieval (pp. 271–278). ACM.

  • Zhang, G., Kan, M., Shan, S., & Chen, X. (2018a). Generative adversarial network with spatial attention for face attribute editing. In Proceedings of the European conference on computer vision (ECCV) (pp. 417–432).

  • Zhang, J., Shu, Y., Xu, S., Cao, G., Zhong, F., Liu, M., & Qin, X. (2018b). Sparsely grouped multi-task generative adversarial networks for facial attribute manipulation. In ACM Multimedia conference on multimedia conference (ACMMM) (pp. 392–401). ACM.

  • Zhang, J., Zhong, F., Cao, G., & Qin, X. (2017a). ST-GAN: Unsupervised facial image semantic transformation using generative adversarial networks. In Proceedings of the Asian conference on machine learning (ACML) (pp. 248–263).

  • Zhang, N., Paluri, M., Ranzato, M., Darrell, T., & Bourdev, L. (2014). Panda: Pose aligned networks for deep attribute modeling. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1637–1644). IEEE.

  • Zhang, S., He, R., Sun, Z., & Tan, T. (2018c). Demeshnet: Blind face inpainting for deep meshface verification. IEEE Transactions on Information Forensics and Security (TIFS), 13(3), 637–647.

    Article  Google Scholar 

  • Zhang, Y. (2018). Xogan: One-to-many unsupervised image-to-image translation. arXiv preprint arXiv:1805.07277.

  • Zhang, Z., Song, Y., & Qi, H. (2017b). Age progression/regression by conditional adversarial autoencoder. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 4352–4360).

  • Zhong, Y., Sullivan, J., & Li, H. (2016a). Face attribute prediction using off-the-shelf CNN features. In Proceedings of the IEEE international conference on biometrics (ICB) (pp. 1–7). IEEE.

  • Zhong, Y., Sullivan, J., & Li, H. (2016b). Leveraging mid-level deep representations for predicting face attributes in the wild. In Proceedings of the IEEE international conference on image processing (ICIP) (pp. 3239–3243). IEEE.

  • Zhou, S., Xiao, T., Yang, Y., Feng, D., He, Q., & He, W. (2017). Genegan: Learning object transfiguration and attribute subspace from unpaired data. In Proceedings of the British machine vision conference (BMVC).

  • Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 2242–2251).

  • Zhuang, N., Yan, Y., Chen, S., & Wang, H. (2018). Multi-task learning of cascaded CNN for facial attribute classification. In Proceedings of the international conference on pattern recognition (ICPR) (pp. 2069–2074). IEEE.

  • Zhuang, N., Yan, Y., Chen, S., Wang, H., & Shen, C. (2018). Multi-label learning based deep transfer neural network for facial attribute classification. Pattern Recognition, 80, 225–240.

    Article  Google Scholar 

Download references

Acknowledgements

We thank the contributions of pioneer researchers in the field of deep learning based facial attribute analysis and other related fields. This work is supported in part by the State Key Development Program (Grant No. 2016YFB1001001), in part by the National Natural Science Foundation of China (NSFC) under Grant U1736119, and in part by the Fundamental Research Funds for the Central Universities under Grant DUT18JC06.

Author information

Authors and Affiliations

  1. School of Information and Communication Engineering, Dalian University of Technology, Dalian, 116024, China

    Xin Zheng & Yanqing Guo

  2. National Laboratory of Pattern Recognition, CASIA, Beijing, 100190, China

    Huaibo Huang, Yi Li & Ran He

  3. Center for Research on Intelligent Perception and Computing, CASIA, Beijing, 100190, China

    Huaibo Huang, Yi Li & Ran He

  4. University of Chinese Academy of Sciences, Beijing, 100190, China

    Huaibo Huang, Yi Li & Ran He

  5. Center for Excellence in Brain Science and Intelligence Technology, CAS, Beijing, 100190, China

    Ran He

Authors
  1. Xin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanqing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huaibo Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ran He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanqing Guo.

Additional information

Communicated by Li Liu, Matti Pietikäinen, Jie Qin, Jie Chen, Wanli Ouyang, Luc Van Gool.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, X., Guo, Y., Huang, H. et al. A Survey of Deep Facial Attribute Analysis. Int J Comput Vis 128, 2002–2034 (2020). https://doi.org/10.1007/s11263-020-01308-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-020-01308-z

Keywords

Search

Navigation