Skip to main content
SpringerLink
Log in

Deflated manifold embedding PCA framework via multiple instance factorings

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Principal component analysis is a widely used technique. However, it is sensitive to noise and considers data samples to be linearly distributed globally. To tackle these challenges, a novel technique robust to noise termed deflated manifold embedding PCA is proposed. In this framework, we unify PCA with manifold embedding to preserve both global and local geometric structures of linear and non-linear data in sub-manifolds. Additionally, a scaling-factor is imposed in the instance space to mitigate the impact of noise in pursuing projections. By using cosine similarity and total distance approaches, we iteratively learn the relationships between instances and projections in order to discriminate between authentic and corrupt instances. Further, a deflation technique is applied to establish multi-relationships between instances and every pursued projection for thorough discrimination. Experimental evaluation of the proposed methods on five datasets show great improvements in their performances over six state-of-the-art techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html

  2. http://www.face-rec.org/databases/

  3. http://yann.lecun.com/exdb/mnist/

  4. http://www1.cs.columbia.edu/CAVE/publications/pdfs/Nene_TR96.pdf

  5. http://www.vision.caltech.edu/Image_Datasets/Caltech101/Caltech101.html

References

  1. Abeo TA, Shen X-J, Bao B-K, Zha Z-J, Fan J (2019) A generalized multi-dictionary least squares framework regularized with multi-graph embeddings. Pattern Recogn 90:1–11

    Google Scholar 

  2. Aminu M, Ahmad NA (2019) Feature extraction using discriminant graph laplacian principal component analysis with application to biomedical datasets. In: Journal of Physics: Conference Series. IOP Publishing, pp 012002

  3. Aïssa-el-bey A, Seghouane A-K (2017) Sparse and smooth canonical correlation analysis through rank-1 matrix approximation. EURASIP J Adv Signal Process 2017(1):25

    Google Scholar 

  4. Becker H, Albera L, Comon P, Kachenoura A, Merlet I (2015) A penalized semialgebraic deflation ica algorithm for the efficient extraction of interictal epileptic signals. IEEE J Biomed Health Inf 21(1):94–104

    Google Scholar 

  5. Cai D, He X, Zhou K, Han J, Bao H (2007) Locality sensitive discriminant analysis.. In: IJCAI, vol 2007, pp 1713–1726

  6. Chaib S, Gu Y, Yao H (2015) An informative feature selection method based on sparse pca for vhr scene classification. IEEE Geosci Remote Sens Lett 13(2):147–151

    Google Scholar 

  7. Chen J, Ye J, Li Q (2007) Integrating global and local structures: A least squares framework for dimensionality reduction. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 1–8

  8. Chen S-B, Ding C, Zhou Z-L, Luo B (2019) Feature selection based on correlation deflation. Neural Comput Appl 31(10):6383–6392

    Google Scholar 

  9. Cherapanamjeri Y, Jain P, Netrapalli P (2017) Thresholding based efficient outlier robust pca, arXiv:1702.05571, pp 1–30

  10. Datta A, Ghosh S, Ghosh A (2018) Pca, kernel pca and dimensionality reduction in hyperspectral images. In: Advances in Principal Component Analysis. Springer, pp 19–46

  11. Ding C, Zhou D, He X, Zha H (2006) R 1-pca: rotational invariant l 1-norm principal component analysis for robust subspace factorization. In: Proceedings of the 23rd international conference on Machine learning. pp 281–288

  12. Eckart C, Young G (1936) The approximation of one matrix by another of lower rank. Psychometrika 1(3):211–218

    MATH  Google Scholar 

  13. Feng G, Hu D, Zhou Z (2008) A direct locality preserving projections (dlpp) algorithm for image recognition. Neural Process Lett 27(3):247–255

    Google Scholar 

  14. Feng C-M, Gao Y-L, Liu J-X, Wang J, Wang D-Q, Wen C-G (2017) Joint-norm constraint and graph-laplacian pca method for feature extraction. BioMed Research International

  15. Goodenough DG, Han T (2009) Reducing noise in hyperspectal data—a nonlinear data series analysis approach. In: 2009 first workshop on hyperspectral image and signal processing, Evolution in Remote Sensing. IEEE, pp 1–4

  16. Hansen TJ, Abrahamsen TJ, Hansen LK (2014) Denoising by semi-supervised kernel pca preimaging. Pattern Recogn Lett 49:114–120

    Google Scholar 

  17. He J, Bi Y, Liu B, Zeng Z (2019) Graph-dual laplacian principal component analysis. J Ambient Intell Human Comput 10(8):3249–3262

    Google Scholar 

  18. He Z, Wu J, Han N (2020) Flexible robust principal component analysis. Int J Mach Learn Cybern 11(3):603–613

    Google Scholar 

  19. Huang P, Gao G (2015) Local similarity preserving projections for face recognition. AEU-Int J Electron Commun 69(11):1724–1732

    Google Scholar 

  20. Huang S, Yang D, Zhou J, Zhang X (2015) Graph regularized linear discriminant analysis and its generalization. Pattern Anal Appl 18 (3):639–650

    MathSciNet  Google Scholar 

  21. Huang K-K, Dai D-Q, Ren C-X (2017) Regularized coplanar discriminant analysis for dimensionality reduction. Pattern Recogn 62:87–98

    MATH  Google Scholar 

  22. Jiang B, Ding C, Luo B, Tang J (2013) Graph-laplacian pca: Closed-form solution and robustness. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3492–3498

  23. Jiang B, Ding C, Luo B (2018) Robust data representation using locally linear embedding guided pca. Neurocomputing 275:523–532

    Google Scholar 

  24. Karami A, Yazdi M, Asli AZ (2011) Noise reduction of hyperspectral images using kernel non-negative tucker decomposition. IEEE J Sel Top Signal Process 5(3):487–493

    Google Scholar 

  25. Karami A, Tafakori L (2017) Image denoising using generalised cauchy filter. IET Image Process 11(9):767–776

    Google Scholar 

  26. Khanna R, Ghosh J, Poldrack R, Koyejo O (2017) A deflation method for structured probabilistic pca. In: Proceedings of the 2017 SIAM International Conference on Data Mining. SIAM, pp 534–542

  27. Koringa PA, Mitra SK (2019) L1-norm orthogonal neighbourhood preserving projection and its applications. Pattern Anal Appl 22(4):1481–1492

    MathSciNet  Google Scholar 

  28. Lai Z, Xu Y, Yang J, Shen L, Zhang D (2016) Rotational invariant dimensionality reduction algorithms. IEEE Trans Cybern 47(11):3733–3746

    Google Scholar 

  29. Leng L, Zhang J, Chen G, Khan MK, Alghathbar K (2011) Two-directional two-dimensional random projection and its variations for face and palmprint recognition. In: International conference on computational science and its applications. Springer, pp 458–470

  30. Leng L, Zhang S, Bi X, Khan MK (2012) Two-dimensional cancelable biometric scheme. In: 2012 International Conference on Wavelet Analysis and Pattern Recognition. IEEE, pp 164–169

  31. Leng L, Li M, Kim C, Bi X (2017) Dual-source discrimination power analysis for multi-instance contactless palmprint recognition. Multimed Tools Appl 76(1):333–354

    Google Scholar 

  32. Leng L, Zhang J, Xu J, Khan MK, Alghathbar K (2010) Dynamic weighted discrimination power analysis in dct domain for face and palmprint recognition. In: 2010 international conference on information and communication technology convergence (ICTC). IEEE, pp 467–471

  33. Li Y, He Z (2017) Robust principal component analysis via feature self-representation. In: 2017 international conference on security, pattern analysis, and cybernetics (SPAC). IEEE, pp 94–99

  34. Liang Z, Xia S, Zhou Y, Zhang L, Li Y (2013) Feature extraction based on lp-norm generalized principal component analysis. Pattern Recogn Lett 34(9):1037–1045

    Google Scholar 

  35. Liu X, Tosun D, Weiner MW, Schuff N, Initiative ADN et al (2013) Locally linear embedding (lle) for mri based alzheimer’s disease classification. Neuroimage 83:148–157

    Google Scholar 

  36. Liu Y, Gao X, Gao Q, Shao L, Han J (2019) Adaptive robust principal component analysis. Neural Netw 119:85–92

    MATH  Google Scholar 

  37. López-Rubio FJ, Lopez-Rubio E, Molina-Cabello MA, Luque-Baena RM, Palomo EJ, Dominguez E (2018) The effect of noise on foreground detection algorithms. Artif Intell Rev 49(3):407–438

    Google Scholar 

  38. Luo M, Nie F, Chang X, Yang Y, Hauptmann A, Zheng Q (2016) Avoiding optimal mean robust pca/2dpca with non-greedy l1-norm maximization. In: Proceedings of International Joint Conference on Artificial Intelligence, pp 1802–1808

  39. Menon V, Kalyani S (2018) Fast, parameter free outlier identification for robust pca, arXiv:1804.04791, pp 1–13

  40. Mi J-X, Zhu Q, Lu J (2019) Principal component analysis based on block-norm minimization. Appl Intell 49(6):2169–2177

    Google Scholar 

  41. Monteiro JM, Rao A, Shawe-Taylor J, Mourao-miranda J, Initiative AD et al (2016) A multiple hold-out framework for sparse partial least squares. J Neurosci Methods 271:182–194

    Google Scholar 

  42. Narayanan RM, Ponnappan SK, Reichenbach SE (2003) Effects of noise on the information content of remote sensing images. Geocarto Int 18(2):15–26

    Google Scholar 

  43. Nie F, Yuan J, Huang H (2014) Optimal mean robust principal component analysis. In: International conference on machine learning, pp 1062–1070

  44. Oh J, Kwak N (2016) Generalized mean for robust principal component analysis. Pattern Recogn 54:116–127

    Google Scholar 

  45. Pan Y, Zhou Y, Liu W, Nie L (2019) Principal component analysis on graph-hessian. In: 2019 IEEE symposium series on computational intelligence (SSCI). IEEE, pp 1494–1501

  46. Park CH, Park H (2005) Nonlinear discriminant analysis using kernel functions and the generalized singular value decomposition. SIAM J Matrix Anal Appl 27(1):87–102

    MathSciNet  MATH  Google Scholar 

  47. Qian L, Zhang L, Bao X, Li F, Yang J (2016) Supervised sparse neighbourhood preserving embedding. IET Image Process 11(3):190–199

    Google Scholar 

  48. Ravi S, Mankame DP, Nayeem S (2013) Face recognition using pca and lda: Analysis and comparison. In: Fifth International Conference on Advances in Recent Technologies in Communication and Computing (ARTCom 2013). IET, pp 6–16

  49. Shaw B, Jebara T (2009) Structure preserving embedding. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp 937–944

  50. Shen X-J, Liu S-X, Bao B-K, Pan C-H, Zha Z-J, Fan J (2020) A generalized least-squares approach regularized with graph embedding for dimensionality reduction. Pattern Recogn 98:107023

  51. Shi Q, Lu H, Cheung Y-M (2017) Rank-one matrix completion with automatic rank estimation via l1-norm regularization. IEEE Trans Neural Netw Learn Syst 29(10):4744–4757

    MathSciNet  Google Scholar 

  52. Tang M, Nie F, Jain R (2017) A graph regularized dimension reduction method for out-of-sample data. Neurocomputing 225:58–63

    Google Scholar 

  53. Thang ND, Lee Y-K, Lee S, et al. (2013) Deflation-based power iteration clustering. Appl Intell 39(2):367–385

    Google Scholar 

  54. Tsai FS (2011) Dimensionality reduction techniques for blog visualization. Expert Syst Appl 38(3):2766–2773

    Google Scholar 

  55. Tu ST, Chen JY, Yang W, Sun H (2011) Laplacian eigenmaps-based polarimetric dimensionality reduction for sar image classification. IEEE Trans Geosci Remote Sens 50(1):170–179

    Google Scholar 

  56. Vaswani N, Bouwmans T, Javed S, Narayanamurthy P (2018) Robust subspace learning: Robust pca, robust subspace tracking, and robust subspace recovery. IEEE Signal Process Mag 35(4):32–55

    Google Scholar 

  57. Vidal R, Ma Y, Sastry S (2016) Principal component analysis. In: Generalized principal component analysis, vol 40. Springer, New York

  58. Wang J (2012) Geometric structure of high-dimensional data and dimensionality reduction. In: Geometric structure of high-dimensional data and dimensionality reduction. Springer, Berlin, pp 51–77

  59. Wang J (2015) Generalized 2-d principal component analysis by lp-norm for image analysis. IEEE Trans Cybern 46(3):792–803

    Google Scholar 

  60. Wang Q, Wang W, Nian R, He B, Shen Y, Björk K-M, Lendasse A (2016) Manifold learning in local tangent space via extreme learning machine. Neurocomputing 174:18–30

    Google Scholar 

  61. Wang S, Xie D, Chen F, Gao Q (2018) Dimensionality reduction by lpp-l21. IET Comput Vis 12(5):659–665

    Google Scholar 

  62. Wang D, Tanaka T (2020) Robust kernel principal component analysis with2, 1-regularized loss minimization. IEEE Access 8:81864–81875

    Google Scholar 

  63. Wen J, Fang X, Cui J, Fei L, Yan K, Chen Y, Xu Y (2018) Robust sparse linear discriminant analysis. IEEE Trans Circ Syst Video Technol 29(2):390–403

    Google Scholar 

  64. Wornyo DK, Shen X-J, Dong Y, Wang L, Huang S-C (2019) Co-regularized kernel ensemble regression. World Wide Web 22(2):717–734

    Google Scholar 

  65. Xing W, Shah AA, Nair PB (2015) Reduced dimensional gaussian process emulators of parametrized partial differential equations based on isomap, Proceedings of the Royal Society A: Mathematical. Phys Eng Sci 471(2174):20140697

  66. Xu J, Yang J (2009) Local graph embedding discriminant analysis for face recognition with single training sample per person. In: 2009 Chinese Conference on Pattern Recognition. IEEE, pp 1–5

  67. Yang L, Liu X, Nie F, Liu Y (2020) Robust and efficient linear discriminant analysis with l 2, 1-norm for feature selection. IEEE Access 8:44100–44110

    Google Scholar 

  68. Ye Q, Fu L, Zhang Z, Zhao H, Naiem M (2018) Lp-and ls-norm distance based robust linear discriminant analysis. Neural Netw 105:393–404

    MATH  Google Scholar 

  69. Yi S, He Z, yang W-J (2017) Robust principal component analysis via joint 2, 1-norms minimization. In: 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC) IEEE, pp 13–18

  70. Yi S, Lai Z, He Z, Cheung Y-m, Liu Y (2017) Joint sparse principal component analysis. Pattern Recogn 61:524–536

    MATH  Google Scholar 

  71. Yi S, He Z, Jing X-Y, Li Y, Cheung Y-M, Nie F (2019) Adaptive weighted sparse principal component analysis for robust unsupervised feature selection. IEEE Trans Neural Netw Learn Syst 31(6):2153–2163

  72. Zhang C, Nie F, Xiang S (2010) A general kernelization framework for learning algorithms based on kernel pca. Neurocomputing 73(4-6):959–967

    Google Scholar 

  73. Zhao D, Lin Z, Tang X (2007) Laplacian pca and its applications. In: 2007 IEEE 11th International Conference on Computer Vision. IEEE, pp 1–8

  74. Zhou Y, Ding Y, Luo Y, Ren H (2016) Sparse neighborhood preserving embedding via l2, 1-norm minimization. In: 2016 9th International Symposium on Computational Intelligence and Design (ISCID), vol 2. IEEE, pp 378–382

Download references

Acknowledgments

This work was funded by the National Natural Science Foundation of China, No. 61572240.

Author information

Authors and Affiliations

  1. School of Computer Science and Communication Engineering, Jiangsu University, Zhenjiang, 212013, China

    Ernest Domanaanmwi Ganaa & Xiang-Jun Shen

  2. School of Applied Science and Technology, Wa Technical University, Box 553, Wa, Ghana

    Ernest Domanaanmwi Ganaa

  3. Jingkou New-Generation Information Technology Industry Institute, JiangSu University, JiangSu, China

    Xiang-Jun Shen

  4. School of Applied Science, Tamale Technical University, Box 3ER, Tamale, Ghana

    Timothy Apasiba Abeo

Authors
  1. Ernest Domanaanmwi Ganaa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang-Jun Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Timothy Apasiba Abeo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiang-Jun Shen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ganaa, E.D., Shen, XJ. & Abeo, T.A. Deflated manifold embedding PCA framework via multiple instance factorings. Multimed Tools Appl 80, 3809–3833 (2021). https://doi.org/10.1007/s11042-020-09789-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09789-3

Keywords

Search

Navigation