Skip to main content
Log in

Bayesian Robust Principal Component Analysis with Adaptive Singular Value Penalty

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Robust principal component analysis (RPCA) has recently seen ubiquitous activity for dimensionality reduction in image processing, visualization and pattern recognition. Conventional RPCA methods model the low-rank component as regularizing each singular value equally. However, in numerous modern applications, each singular value has different physical meaning and should be treated differently. This is one of the main reasons why RPCA techniques cannot work well in dealing with many realistic problems. To solve this problem, a novel hierarchical Bayesian RPCA model with adaptive singular value penalty is proposed. This model enforces the low-rank constraint by introducing an adaptive penalty function on the singular values of the low-rank component. In particular, we impose a hierarchical Exponent-Gamma prior on the singular values of the low-rank component and the Beta-Bernoulli prior on sparsity indicators. The variational Bayesian framework and the Markov chain Monte Carlo-based Bayesian inference are considered for inferring the posteriors of all latent variables involved in low-rank and sparse components. Numerical experiments demonstrate the competitive performance of the proposed model on synthetic and real data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. S.D. Babacan, M. Luessi, R. Molina, A.K. Katsaggelos, Sparse Bayesian methods for low-rank matrix estimation. IEEE Trans. Signal Process. 60(8), 3964–3977 (2012)

    Article  MathSciNet  Google Scholar 

  2. S.D. Babacan, S. Nakajima, M.N. Do, Bayesian group-sparse modeling and variational inference. IEEE Trans. Signal Process. 62(11), 2906–2921 (2015)

    Article  MathSciNet  Google Scholar 

  3. C.M. Bishop, Pattern Recognition and Machine Learning (Springer, Berlin, 2006)

    MATH  Google Scholar 

  4. J. Cai, E.J. Candès, Z. Shen, A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2008)

    Article  MathSciNet  Google Scholar 

  5. E.J. Candès, X. Li, Y. Ma, J. Wright, Robust principal component analysis? J. ACM 58(3), 11 (2009)

    MathSciNet  MATH  Google Scholar 

  6. E.J. Candès, M.B. Wakin, S.P. Boyd, Enhancing sparsity by reweighted \(\ell _{1}\) minimization. J. Fourier Anal. Appl. 14(5–6), 877–905 (2008)

    Article  MathSciNet  Google Scholar 

  7. X. Cao, Y. Chen, Q. Zhao, D. Meng, Y. Wang, D. Wang, Z. Xu, Low-rank matrix factorization under general mixture noise distributions, in IEEE International Conference on Computer Vision, (2016), pp. 1493–1501

  8. G.C. Cawley, N.L.C. Talbot, Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters. J. Mach. Learn. Res. 8, 841–861 (2007)

    MATH  Google Scholar 

  9. F. Chen, R. Hu, H. Yu, S. Wang, Reduced set density estimator for object segmentation based on shape probabilistic representation. J. Vis. Commun. Image Represent. 23(7), 1085–1094 (2012)

    Article  Google Scholar 

  10. Y. Chen, X. Cao, Q. Zhao, D. Meng, Z. Xu, Denoising hyperspectral image with non-i.i.d. noise structure. IEEE Trans. Cybern. 48(3), 1054–1066 (2018)

    Article  Google Scholar 

  11. X. Ding, L. He, L. Carin, Bayesian robust principal component analysis. IEEE Trans. Image Process. 20(12), 3419–3430 (2011)

    Article  MathSciNet  Google Scholar 

  12. Z. Fan, X. Yong, W. Zuo, Y. Jian, J. Tang, Z. Lai, D. Zhang, Modified principal component analysis: an integration of multiple similarity subspace models. IEEE Trans. Neural Netw. Learn. Syst. 25(8), 1538–1552 (2017)

    Article  Google Scholar 

  13. J. Gao, Robust \(l_1\) principal component analysis and its Bayesian variational inference. Neural Comput. 20(2), 555–578 (2008)

    Article  MathSciNet  Google Scholar 

  14. A. Ghaani Farashahi, Cyclic wave packet transform on finite abelian groups of prime order. Int. J. Wavelets Multiresolut. Inf. Process. 12(06), 1450041 (2014)

    Article  MathSciNet  Google Scholar 

  15. A. Ghaani Farashahi, Wave packet transforms over finite cyclic groups. Linear Algebra Appl. 489, 75–92 (2016)

    Article  MathSciNet  Google Scholar 

  16. P. Giordani, H.A.L. Kiers, A comparison of three methods for principal component analysis of fuzzy interval data. Comput. Stat. Data Anal. 51(1), 379–397 (2006)

    Article  MathSciNet  Google Scholar 

  17. S. Gu, Q. Xie, D. Meng, W. Zuo, X. Feng, L. Zhang, Weighted nuclear norm minimization and its applications to low level vision. Int. J. Comput. Vis. 121(2), 183–208 (2017)

    Article  Google Scholar 

  18. G. Han, J. Wang, X. Cai, Background subtraction based on modified online robust principal component analysis. Int. J. Mach. Learn. Cybern. 8(6), 1839–1852 (2017)

    Article  Google Scholar 

  19. N. Han, Y. Song, Z. Song, Bayesian robust principal component analysis with structured sparse component. Comput. Stat. Data Anal. 109, 144–158 (2017)

    Article  MathSciNet  Google Scholar 

  20. N. Han, Z. Song, Bayesian multiple measurement vector problem with spatial structured sparsity patterns. Digit. Signal Process. 75, 184–201 (2018)

    Article  MathSciNet  Google Scholar 

  21. N. Han, Z. Song, Y. Li, Cluster-based image super-resolution via jointly low-rank and sparse representation. J. Vis. Commun. Image Represent. 38, 175–185 (2016)

    Article  Google Scholar 

  22. Y. Hu, D. Zhang, J. Ye, X. Li, X. He, Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2117–2130 (2013)

    Article  Google Scholar 

  23. O.Y. Lee, J.W. Lee, J.O. Kim, Combining self-learning based super-resolution with denoising for noisy images. J. Vis. Commun. Image Represent. 48, 66–76 (2017)

    Article  Google Scholar 

  24. Z. Lin, M. Chen, Y. Ma, The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055 (2010)

  25. G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, Y. Ma, Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)

    Article  Google Scholar 

  26. S. Liu, H. Wu, Y. Huang, Y. Yang, J. Jia, Accelerated structure-aware sparse Bayesian learning for 3D electrical impedance tomography. IEEE Trans. Ind. Inform. 15(9), 5033–5041 (2019)

    Article  Google Scholar 

  27. S. Liu, Y.D. Zhang, T. Shan, R. Tao, Structure-aware Bayesian compressive sensing for frequency-hopping spectrum estimation with missing observations. IEEE Trans. Signal Process. 66(8), 2153–2166 (2018)

    Article  MathSciNet  Google Scholar 

  28. C. Lu, J. Tang, S. Yan, Z. Lin, Nonconvex nonsmooth low-rank minimization via iteratively reweighted nuclear norm. IEEE Trans. Image Process. 25(2), 829–839 (2016)

    Article  MathSciNet  Google Scholar 

  29. J. Luttinen, A. Ilin, J. Karhunen, Bayesian robust \(pca\) for incomplete data, in International Conference on Independent Component Analysis and Signal Separation, (2009), pp. 66–73

  30. A.B. Musa, A comparison of \(\ell _1\)-regularization, PCA, KPCA and ICA for dimensionality reduction in logistic regression. Int. J. Mach. Learn. Cybern. 5(6), 861–873 (2014)

    Article  Google Scholar 

  31. N. Nedic, D. Prsic, L. Dubonjic, V. Stojanovic, V. Djordjevic, Optimal cascade hydraulic control for a parallel robot platform by PSO. Int. J. Adv. Manuf. Technol. 72(5–8), 1085–1098 (2014)

    Article  Google Scholar 

  32. N. Nedic, D. Prsic, C. Fragassa, V. Stojanovic, A. Pavlovic, Simulation of hydraulic check valve for forestry equipment. Int. J. Heavy Veh. Syst. 24(3), 260–276 (2017)

    Article  Google Scholar 

  33. D. Prsic, N. Nedic, V. Stojanovic, A nature inspired optimal control of pneumatic-driven parallel robot platform. Proc. Inst. Mech. Eng. Part C J. Mech. Eng. Sci. 231(1), 59–71 (2017)

    Article  Google Scholar 

  34. S. Serneels, T. Verdonck, Principal component analysis for data containing outliers and missing elements. Comput. Stat. Data Anal. 52(3), 1712–1727 (2008)

    Article  MathSciNet  Google Scholar 

  35. A. Sharma, K. Paliwal, S. Imoto, S. Miyano, Principal component analysis using QR decomposition. Int. J. Mach. Learn. Cybern. 4(6), 679–683 (2013)

    Article  Google Scholar 

  36. V. Stojanovic, V. Filipovic, Adaptive input design for identification of output error model with constrained output. Circuits Syst. Signal Process. 33(1), 97–113 (2014)

    Article  MathSciNet  Google Scholar 

  37. V. Stojanovic, N. Nedic, D. Prsic, L. Dubonjic, Optimal experiment design for identification of ARX models with constrained output in non-Gaussian noise. Appl. Math. Model. 40(13–14), 6676–6689 (2016)

    Article  MathSciNet  Google Scholar 

  38. K.C. Toh, S. Yun, An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pac. J. Optim. 6(3), 615–640 (2011)

    MathSciNet  MATH  Google Scholar 

  39. A. Wagner, J. Wright, A. Ganesh, Z. Zhou, H. Mobahi, Y. Ma, Toward a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 372–386 (2011)

    Article  Google Scholar 

  40. S. Yi, Z. Lai, Z. He, Y. Liu, Joint sparse principal component analysis. Pattern Recognit. 61, 524–536 (2017)

    Article  Google Scholar 

  41. Z. Zhang, B.D. Rao, Sparse signal recovery with temporally correlated source vectors using sparse Bayesian learning. IEEE J. Sel. Top. Signal Process. 5(5), 912–926 (2011)

    Article  Google Scholar 

  42. Q. Zhao, D. Meng, Z. Xu et al., \(l_{1}\)-norm low-rank matrix factorization by variational \(B\)ayesian method. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 825–839 (2015)

    Article  MathSciNet  Google Scholar 

  43. Q. Zhao, D. Meng, Z. Xu, W. Zuo, L. Zhang, Robust principal component analysis with complex noise, in International Conference on Machine Learning, (2014), pp. 55–63

Download references

Acknowledgements

The authors would like to express their sincere gratitude to the anonymous referees, the editor for many valuable suggestions and comments that helped to improve the paper. This work was supported by the National Natural Science Foundation of China (Grant No. 91746107) and the China Scholarship Council.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Guan Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cui, K., Wang, G., Song, Z. et al. Bayesian Robust Principal Component Analysis with Adaptive Singular Value Penalty. Circuits Syst Signal Process 39, 4110–4135 (2020). https://doi.org/10.1007/s00034-020-01358-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-020-01358-1

Keywords

Navigation