Skip to main content
SpringerLink
Log in

Probabilistic collaborative representation on Grassmann manifold for image set classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

For image-set based classification, sparse coding and collaborative representation have gained a lot of attention due to their robustness and effectiveness. However, most existing methods focus on collaborative representation in Euclidean space. It still remains a research gap to handle this problem from Geometry-Aware perspective and interpret the mechanism of collaborative representation on nonlinear manifold. In this paper, we propose a novel method named probabilistic collaborative representation on Grassmann manifold for image set classification, which is interpreted from a probabilistic viewpoint. Specifically, we regard each image set as a point on Grassmann manifold inspired by its non-Euclidean geometry and then perform collaborative representation on the space of symmetric matrices, which enables us to explain the internal mechanism of classification and derive a closed form solution. Moreover, classification criterion is designed to further improve the performance of the proposed method. Experimental results on four databases (i.e. Honda/UCSD, YaleB, Youtube Celebrities and ETH-80) for face recognition task and object recognition task demonstrate the robustness and effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ma M, Shao M, Zhao X, Fu Y (2013) Prototype based feature learning for face image set classification. In: 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–6

  2. Zhang Q, Sun H (2018) Probabilistic collaborative representation based orthogonal discriminative projection for image set classification. J Vis Commun Image Represent 55:106–114

    Article  Google Scholar 

  3. Kim T-K, Kittler J, Cipolla R (2007) Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans Pattern Anal Mach Intell 29(6):1005–1018

    Article  Google Scholar 

  4. Wang R, Chen X (2009) Manifold discriminant analysis. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 429–436

  5. Guo Y, He R, Zheng W-S, Kong X, He Z (2013) Robust spectral regression for face recognition. Neurocomputing 118:33–40

    Article  Google Scholar 

  6. Lu J, Liong VE, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056

    Article  Google Scholar 

  7. Ren Z, Bin W, Zhang X, Sun Q (2019) Image set classification using candidate sets selection and improved reverse training. Neurocomputing 341:60–69

    Article  Google Scholar 

  8. Ortiz EG, Wright A, Shah M (2013) Face recognition in movie trailers via mean sequence sparse representation-based classification. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3531–3538

  9. Hajati F, Tavakolian M, Gheisari S, Gao Y, Mian AS (2017) Dynamic texture comparison using derivative sparse representation: application to video-based face recognition. IEEE Trans Human Mach Syst 47(6):970–982

    Article  Google Scholar 

  10. Chen Y-C, Patel VM, Shekhar S, Chellappa R, Phillips PJ (2013) Video-based face recognition via joint sparse representation. In: 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–8

  11. Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: which helps face recognition? In: IEEE international conference on computer vision (ICCV). IEEE, pp 471–478

  12. Zhu P, Zuo W, Zhang L, Chi-Keung Shiu S, Zhang D (2014) Image set-based collaborative representation for face recognition. IEEE Trans Inf Forensics Secur 9(7):1120–1132

    Article  Google Scholar 

  13. Harandi M, Sanderson C, Shen C, Lovell BC (2013) Dictionary learning and sparse coding on grassmann manifolds: an extrinsic solution. In: IEEE international conference on computer vision (ICCV). IEEE, pp 3120–3127

  14. Chen S, Sanderson C, Harandi MT, Lovell BC (2013) Improved image set classification via joint sparse approximated nearest subspaces. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 452–459

  15. Yamaguchi O, Fukui K, Maeda K (1998) Face recognition using temporal image sequence. In: 3th IEEE international conference on automatic face and gesture recognition (FG). IEEE, pp 318–323

  16. Huang Z, Wang R, Shan S, Chen X (2015) Projection metric learning on grassmann manifold with application to video based face recognition. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 140–149

  17. Gao X, Sun Q, Haitao X, Wei D, Gao J (2019) Multi-model fusion metric learning for image set classification. Knowl Based Syst 164:253–264

    Article  Google Scholar 

  18. Hamm J, Lee DD (2008) Grassmann discriminant analysis: a unifying view on subspace-based learning. In: 25th International conference on machine learning (ICML). ACM, pp 376–383

  19. Cai S, Zhang L, Zuo W, Feng X (2016) A probabilistic collaborative representation based approach for pattern classification. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2950–2959

  20. Acharya UR, Chowriappa P, Fujita H, Bhat S, Dua S, Koh JEW, Eugene LWJ, Kongmebhol P, Ng KH (2016) Thyroid lesion classification in 242 patient population using gabor transform features from high resolution ultrasound images. Knowl Based Syst 107:235–245

    Article  Google Scholar 

  21. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  22. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

  23. Sun Q-S, Zeng S-G, Liu Y, Heng P-A, Xia D-S (2005) A new method of feature fusion and its application in image recognition. Pattern Recogn 38(12):2437–2448

    Article  Google Scholar 

  24. Wang R, Shan S, Chen X, Gao W (2008) Manifold-manifold distance with application to face recognition based on image set. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–8

  25. Huang Z, Wang R, Shan S, Chen X (2014) Learning Euclidean-to-Riemannian metric for point-to-set classification. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1677–1684

  26. Cevikalp H, Triggs B (2010) Face recognition based on image sets. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2567–2573

  27. Hu Y, Mian AS, Owens R (2011) Sparse approximated nearest points for image set classification. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 121–128

  28. Caseiro R, Martins P, Henriques JF, Silva LF, Batista J (2013) Rolling Riemannian manifolds to solve the multi-class classification problem. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 41–48

  29. Wang R, Guo H, Davis LS, Dai Q (2012) Covariance discriminative learning: a natural and efficient approach to image set classification. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2496–2503

  30. Harandi MT, Sanderson C, Shirazi S, Lovell BC (2011) Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching. In: CVPR 2011. IEEE, pp 2705–2712

  31. Wei D, Shen X, Sun Q, Gao X, Yan W (2020) Locality-aware group sparse coding on grassmann manifolds for image set classification. Neurocomputing 385:197–210

    Article  Google Scholar 

  32. Wei D, Shen X, Sun Q, Gao X, Yan W (2020) Prototype learning and collaborative representation using grassmann manifolds for image set classification. Pattern Recogn 100:107123

    Article  Google Scholar 

  33. Zhang H, Wang S, Zhao M, Xiaofei X, Ye Y (2018) Locality reconstruction models for book representation. IEEE Trans Knowl Data Eng 30(10):1873–1886

    Article  Google Scholar 

  34. Cui Z, Chang H, Shan S, Ma B, Chen X (2014) Joint sparse representation for video-based face recognition. Neurocomputing 135:306–312

    Article  Google Scholar 

  35. Absil P-A, Robert M, Rodolphe S (2009) Optimization algorithms on matrix manifolds. Princeton University Press, Princeton

    Google Scholar 

  36. Kim S, Koh K, Lustig M, Boyd S, Gorinevsky D (2007) An interior-point method for large-scale l1-regularized logistic regression. J Mach Learn Res 8:1519–1555

    MathSciNet  MATH  Google Scholar 

  37. Lee K-C, Ho J, Yang M-H, Kriegman D (2003) Video-based face recognition using probabilistic appearance manifolds. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–313

  38. Fukui K, Yamaguchi O (2005) Face recognition using multi-viewpoint patterns for robot vision. In: International symposium of robotics research. Springer, pp 192–201

  39. Kim M, Kumar S, Pavlovic V, Rowley H (2008) Face tracking and recognition with visual constraints in real-world videos. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–8

  40. Kim T-K, Cipolla R (2008) Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Trans Pattern Anal Mach Intell 31(8):1415–1428

    Google Scholar 

  41. Paul V, Jones Michael J (2004) Robust real-time face detection. Int J Comput Vision 57(2):137–154

    Article  Google Scholar 

  42. Yang M, Zhu P, Van Gool L, Zhang L (2013) Face recognition based on regularized nearest points between image sets. In: 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–7

  43. Harandi MT, Sanderson C, Hartley R, Lovell BC (2012) Sparse coding and dictionary learning for symmetric positive definite matrices: a kernel approach. In: European conference on computer vision (ECCV). Springer, pp 216–229

  44. Deng W, Jiani H, Guo J (2017) Face recognition via collaborative representation: Its discriminant nature and superposed representation. IEEE Trans Pattern Anal Mach Intell 40(10):2513–2521

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Science Foundation of China under Grant No. 61673220.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Shuo Zhang, Dong Wei, Wenzhu Yan & Quansen Sun

Authors
  1. Shuo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenzhu Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quansen Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quansen Sun.

Ethics declarations

Conflict of interest

We declare that we have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Wei, D., Yan, W. et al. Probabilistic collaborative representation on Grassmann manifold for image set classification. Neural Comput & Applic 33, 2483–2496 (2021). https://doi.org/10.1007/s00521-020-05089-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05089-x

Keywords

Search

Navigation