Skip to main content

Advertisement

SpringerLink
Log in

Nonnegative Tensor Factorization based on Low-Rank Subspace for Facial Expression Recognition

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Important progresses have been made in the field of artificial intelligence in recent years, and facial expression recognition (FER), which could greatly facilitate the development of human-computer interaction, has been becoming a significant research hotspot. In this paper, a novel nonnegative tensor factorization method is proposed based on low-rank subspace (NTF−LRS) for FER. Firstly, in order to find the high order correlations underlying multi-dimensional data, a data tensor model is constructed, which could represent different dimensional features ingeniously. And then, the low-rank subspace model is adopted to reconstruct the original tensor model, reduce the redundancy of the learned new tensor, and improve the discriminant abilities of inter-class information. Finally, the reconstructed tensor is decomposed to get factor matrices by nonnegative tensor factorization, where all factor matrices are used to extract subspace features. To verify the effectiveness of our proposal, two well-known facial expression datasets named as “JAFFE” and “CK+” are utilized for evaluation, and the experimental results show that the tensor-based method preserves the original structure of whole samples, which avoids the case of dimension curse because of vectorization. In addition, this method uses Laplacian graph to impose regularization on low-rank subspace model, which keeps the local relationship between sample neighbors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Zhang K, Huang Y, Du Y, Wang L (2017) Facial expression recognition based on deep evolutional spatial-temporal networks. IEEE Trans. Image Process. 26(9):4193–4203

    Article  MathSciNet  MATH  Google Scholar 

  2. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: A comprehensive study. Image Vis Comput 27(6):803–816

    Article  Google Scholar 

  3. Shan C, Gong S, McOwan PW (2006) A comprehensive empirical study on linear subspace methods for facial expression analysis. In: IEEE conference on computer vision and pattern recognition workshop. IEEE, pp 153–153

  4. Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 586–591

  5. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence (7):711–720

  6. He X, Cai D, Yan S, Zhang HJ (2005) Neighborhood preserving embedding. In: IEEE international conference on computer vision, vol 2. IEEE, pp 1208–1213

  7. He X, Niyogi P (2004) Locality preserving projections. In: Advances in neural information processing systems, pp 153–160

  8. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788

    Article  MATH  Google Scholar 

  9. Vasilescu MAO, Terzopoulos D (2002) Multilinear analysis of image ensembles: Tensorfaces. In: European conference on computer vision. Springer, Berlin, pp 447–460

  10. Ye J, Janardan R, Li Q (2004) GPCA: an efficient dimension reduction scheme for image compression and retrieval. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 354–363

  11. Tao D, Li X, Wu X, Maybank S (2008) Tensor rank one discriminant analysis—a convergent method for discriminative multilinear subspace selection. Neurocomputing 71(10-12):1866–1882

    Article  Google Scholar 

  12. Dai G, Yeung DY (2006) Tensor embedding methods. National conference on artificial intelligence, vol 6. AAAI, Boston, pp 330–335

    Google Scholar 

  13. Shashua A, Hazan T (2005) Non-negative tensor factorization with applications to statistics and computer vision. In: The 22nd international conference on machine learning. ACM, pp 792–799

  14. Ellison JW, Massaro DW (1997) Featural evaluation, integration, and judgment of facial affect. J. Exp. Psychol. Hum. Percept. Perform. 23(1):213

    Article  Google Scholar 

  15. XiuJun Z, Chang L (2014) Generalized discriminant orthogonal nonnegative tensor factorization for facial expression recognition. The Scientific World Journal. https://doi.org/10.1155/2014/608158

  16. An G, Liu S, Ruan Q (2017) A sparse neighborhood preserving non-negative tensor factorization algorithm for facial expression recognition. Pattern. Anal. Applic. 20(2):453–471

    Article  MathSciNet  Google Scholar 

  17. An G, Liu S, Jin Y, Ruan Q, Lu S (2014) Facial expression recognition based on discriminant neighborhood preserving nonnegative tensor factorization and ELM. Mathematical Problems in Engineering, https://doi.org/10.1155/2014/390328

  18. Fu Y, Ruan Q, Jin Y, An G (2018) Sparse orthogonal tucker decomposition for 2D + 3D facial expression recognition. In: 2018 14th IEEE international conference on signal processing (ICSP). IEEE, pp 516–521

  19. Candès EJ, Li X, Ma Y, Wright J (2011) Robust principal component analysis? Journal of the ACM (JACM) 58(3):11

    Article  MathSciNet  MATH  Google Scholar 

  20. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2012) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  21. Yin M, Gao J, Lin Z (2015) Laplacian regularized low-rank representation and its applications. IEEE Trans Pattern Anal Mach Intell 38(3):504–517

    Article  Google Scholar 

  22. Tang C, Liu X, Zhu X, Xiong J, Li M, Xia J, et al. (2019) Feature selective projection with low-rank embedding and dual Laplacian regularization. IEEE Trans Knowl Data Eng

  23. Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055

  24. Lin Z, Liu R, Su Z (2011) Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in neural information processing systems: 612–620

  25. Cai JF, Candès E J, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Opt 20(4):1956–1982

    Article  MathSciNet  MATH  Google Scholar 

  26. Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. Advances in neural information processing systems: 556–562

  27. Cichocki A, Zdunek R, Amari SI (2007) Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. In: International conference on independent component analysis and signal separation. Springer, Berlin, pp 169–176

  28. Carroll JD, Chang JJ (1970) Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition. Psychometrika 35(3):283–319

    Article  MATH  Google Scholar 

  29. Tucker LR (1966) Some mathematical notes on three-mode factor analysis. Psychometrika 31 (3):279–311

    Article  MathSciNet  Google Scholar 

  30. Zhang S, Ang J, Sun J (2013) An alternating direction method for solving convex nonlinear semidefinite programming problems. Optimization 62(4):527–543

    Article  MathSciNet  MATH  Google Scholar 

  31. Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with gabor wavelets. In: Proceedings third IEEE international conference on automatic face and gesture recognition. IEEE, pp 200–205

  32. Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended Cohn-Kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer society conference on computer vision and pattern recognition-workshops. IEEE, pp 94–101

  33. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  34. Wang Y, Jia Y, Hu C, Turk M (2005) Non-negative matrix factorization framework for face recognition. Int. J. Pattern Recognit. Artif. Intell. 19(04):495–511

    Article  Google Scholar 

  35. Rivera AR, Castillo JR, Chae OO (2012) Local directional number pattern for face analysis: Face and expression recognition. IEEE Trans Image Process 22(5):1740–1752

    Article  MathSciNet  MATH  Google Scholar 

  36. Hamester D, Barros P, Wermter S (2015) Face expression recognition with a 2-channel convolutional neural network. In: 2015 international joint conference on neural networks (IJCNN). IEEE, pp 1–8

  37. Ryu B, Rivera AR, Kim J, Chae O (2017) Local directional ternary pattern for facial expression recognition. IEEE Trans. Image Process. 26(12):6006–6018

    Article  MathSciNet  Google Scholar 

  38. Xie S, Hu H (2018) Facial expression recognition using hierarchical features with deep comprehensive multipatches aggregation convolutional neural networks. IEEE Trans Multimed 21(1):211–220

    Article  Google Scholar 

  39. Wu BF, Lin CH (2018) Adaptive feature mapping for customizing deep learning based facial expression recognition model. IEEE access 6:12451–12461

    Article  Google Scholar 

  40. Liu M, Li S, Shan S, Wang R, Chen X (2014) Deeply learning deformable facial action parts model for dynamic expression analysis. In: Asian conference on computer vision. Springer, Cham, pp 143–157

  41. Wang Z, Wang S, Ji Q (2013) Capturing complex spatio-temporal relations among facial muscles for facial expression recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3422–3429

  42. Liu M, Shan S, Wang R, Chen X (2014) Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1749–1756

  43. Li S, Deng W, Du J (2017) Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2852–2861

  44. Sikka K, Sharma G, Bartlett M (2016) Lomo: Latent ordinal model for facial analysis in videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5580–5589

  45. Cai J, Meng Z, Khan AS, Li Z, O’Reilly J, Tong Y (2018) Island loss for learning discriminative features in facial expression recognition. In: 2018 13th IEEE international conference on automatic face and gesture recognition (FG 2018). IEEE, pp 302–309

  46. Chen Y, Yu L, Ota K, Dong M (2018) Robust activity recognition for aging society. IEEE J Biomed Health Inf 22(6):1754–1764

    Article  Google Scholar 

  47. Li H, Ota K, Dong M, Guo M (2018) Learning human activities through Wi-Fi channel state information with multiple access points. IEEE Commun. Mag. 56(5):124–129

    Article  Google Scholar 

  48. Ota K, Dao MS, Mezaris V, De Natale FG (2017) Deep learning for mobile multimedia: a survey. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 13(3s):34

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Science Foundation China under grant 61872404, the Applied Basic Research Key Programs of Science and Technology Department of Sichuan Province under the grant 2018JY0023, and the National Key Research and Development Program of China under the grant 2018YFB17002402.

Author information

Authors and Affiliations

  1. University of Electronic Science and Technology of China, Chengdu, China

    Xingang Liu, Chenqi Li, Cheng Dai & Jinfeng Lai

  2. National Dong Hwa University, Taiwan, China

    Han-Chieh Chao

Authors
  1. Xingang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheng Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinfeng Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Han-Chieh Chao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xingang Liu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Li, C., Dai, C. et al. Nonnegative Tensor Factorization based on Low-Rank Subspace for Facial Expression Recognition. Mobile Netw Appl 27, 58–69 (2022). https://doi.org/10.1007/s11036-020-01709-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-020-01709-x

Keywords

Search

Navigation