Skip to main content
SpringerLink
Log in

Adaptive sparse and dense hybrid representation with nonconvex optimization

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Sparse representation has been widely used in signal processing, pattern recognition and computer vision etc. Excellent achievements have been made in both theoretical researches and practical applications. However, there are two limitations on the application of classification. One is that sufficient training samples are required for each class, and the other is that samples should be uncorrupted. In order to alleviate above problems, a sparse and dense hybrid representation (SDR) framework has been proposed, where the training dictionary is decomposed into a class-specific dictionary and a non-class-specific dictionary. SDR puts ℓ1 constraint on the coefficients of class-specific dictionary. Nevertheless, it over-emphasizes the sparsity and overlooks the correlation information in class-specific dictionary, which may lead to poor classification results. To overcome this disadvantage, an adaptive sparse and dense hybrid representation with non-convex optimization (ASDR-NO) is proposed in this paper. The trace norm is adopted in class-specific dictionary, which is different from general approaches. By doing so, the dictionary structure becomes adaptive and the representation ability of the dictionary will be improved. Meanwhile, a non-convex surrogate is used to approximate the rank function in dictionary decomposition in order to avoid a suboptimal solution of the original rank minimization, which can be solved by iteratively reweighted nuclear norm (IRNN) algorithm. Extensive experiments conducted on benchmark data sets have verified the effectiveness and advancement of the proposed algorithm compared with the state-of-the-art sparse representation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Natarajan B K. Sparse approximate solutions to linear systems. Siam Journal on Computing, 1995, 24(2): 227–234

    Article  MathSciNet  Google Scholar 

  2. Huang M, Yang W, Jiang J, Wu Y, Zhang Y, Chen W, Feng Q. Brain extraction based on locally linear representation-based classification. Neuroimage, 2014, 92(10): 322–339

    Article  Google Scholar 

  3. Donoho D L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306

    Article  MathSciNet  Google Scholar 

  4. Candès E J, Romberg J, Tao T. Robust uncertainty principles: exact signal frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489–509

    Article  MathSciNet  Google Scholar 

  5. Elad M, Figueiredo M A T, Ma Y. On the role of sparse and redundant representations in image processing. Proceedings of the IEEE, 2010, 98(6): 972–982

    Article  Google Scholar 

  6. Elad M. Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. 1st ed. New York: Springer Science and Business Media, 2010

    Book  Google Scholar 

  7. Bruckstein A M, Donoho D L, Elad M. From sparse solutions of systems of equations to sparse modeling of signals and images. Siam Review, 2009, 51(1): 34–81

    Article  MathSciNet  Google Scholar 

  8. Wright J, Ma Y, Mairal J, Sapiro G, Huang T S, Yan S. Sparse representation for computer vision and pattern recognition. Proceedings of the IEEE, 2010, 98(6): 1031–1044

    Article  Google Scholar 

  9. Candès E, Romberg J. Sparsity and incoherence in compressive sampling. Inverse Problems, 2006, 23(3): 969–985

    Article  MathSciNet  Google Scholar 

  10. Wright J, Yang A Y, Ganesh A, Sastry S S, Ma Y. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 210–227

    Article  Google Scholar 

  11. Cover T, Hart P. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 2002, 13(1): 21–27

    Article  Google Scholar 

  12. Li S, Lu J. Face recognition using the nearest feature line method. IEEE Transactions on Neural Networks, 1999, 10(2): 439–443

    Article  Google Scholar 

  13. Chien J T, Wu C C. Discriminant waveletfaces and nearest feature classifiers for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 24(12): 1644–1649

    Article  Google Scholar 

  14. Lee K C, Ho J, Kriegman D J. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(5): 684–698

    Article  Google Scholar 

  15. Xu Y, Zhu Q, Fan Z, Zhang D, Mi J, Lai Z. Using the idea of the sparse representation to perform coarse-to-fine face recognition. Information Sciences, 2013, 238(7): 138–148

    Article  MathSciNet  Google Scholar 

  16. Yang M, Zhang L. Gabor feature based sparse representation for face recognition with gabor occlusion dictionary. In: Proceedings of European Conference on Computer Vision. 2010, 448–461

  17. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y. Locality-constrained linear coding for image classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2010, 3360–3367

  18. He R, Zheng W S, Hu B G. Maximum correntropy criterion for robust face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(8): 1561–1576

    Google Scholar 

  19. Wagner A, Wright J, Ganesh A, Zhou Z, Mobahi H, Ma Y. Toward a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(2): 372–386

    Article  Google Scholar 

  20. Wang X, Yang M, Shen L. Structured regularized robust coding for face recognition. IEEE Transactions on Image Processing, 2013, 22(5): 1753–1766

    Article  MathSciNet  Google Scholar 

  21. Xu Y, Zhong Z, Yang J, You J, Zhang D. A new discriminative sparse representation method for robust face recognition via l2 regularization. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2233–2242

    Article  MathSciNet  Google Scholar 

  22. Zhang L, Yang M. Sparse representation or collaborative representation: which helps face recognition? In: Proceedings of IEEE International Conference on Computer Vision. 2012, 471–478

  23. Wang J, Lu C, Wang M, Li P, Yan S, Hu X. Robust face recognition via adaptive sparse representation. IEEE Transactions on Cybernetics, 2014, 44(12): 2368–2378

    Article  Google Scholar 

  24. Grave E, Obozinski G, Bach F. Trace lasso: a trace norm regularization for correlated designs. In: Proceedings of the 24th International Conference on Neural Information Processing Systems. 2011, 2187–2195

  25. Candès E J, Li X, Ma Y, Wright J. Robust principal component analysis? Journal of the ACM, 2009, 58(3): 1101–1137

    MathSciNet  MATH  Google Scholar 

  26. Wang Y C F, Wei C P, Chen C F. Low-rank matrix recovery with structural incoherence for robust face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2618–2625

  27. Ma L, Wang C, Xiao B, Zhou W. Sparse representation for face recognition based on discriminative low-rank dictionary learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2586–2593

  28. Zhang Y, Jiang Z, Davis L S. Learning structured low-rank representations for image classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 676–683

  29. Deng W, Hu J, Guo J. Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9): 1864–1870

    Article  Google Scholar 

  30. Jiang X, Lai J. Sparse and dense hybrid representation via dictionary decomposition for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(5): 1067–1079

    Article  Google Scholar 

  31. Yang Y, Ma Z, Hauptmann A G, Sebe N. Feature selection for multimedia analysis by sharing information among multiple tasks. IEEE Transactions on Multimedia, 2013, 15(3): 661–669

    Article  Google Scholar 

  32. Trzasko J, Manduca A. Highly undersampled magnetic resonance image reconstruction via homotopic ℓ0-minimization. IEEE Transactions on Medical Imaging, 2009, 28(1): 106–121

    Article  Google Scholar 

  33. Deng W, Hu J, Guo J. In defense of sparsity based face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 399–406

  34. Donoho D L. For most large underdetermined systems of linear equations the minimal ℓ1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 2010, 59(6): 797–829

    Article  MathSciNet  Google Scholar 

  35. Candès E J, Romberg J K, Tao T. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 2005, 59(8): 1207–1223

    Article  MathSciNet  Google Scholar 

  36. Lin Z, Chen M, Ma Y. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. 2010, arXiv preprint arXiv:1009.5055

  37. Zou H, Hastie T. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, 2005, 67(2): 301–320

    Article  MathSciNet  Google Scholar 

  38. Cai J F, Candès E J, Shen Z. A singular value thresholding algorithm for matrix completion. Siam Journal on Optimization, 2010, 20(4): 1956–1982

    Article  MathSciNet  Google Scholar 

  39. Hale E T, Yin W, Zhang Y. Fixed-point continuation for ℓ1-minimization: methodology and convergence. Siam Journal on Optimization, 2008, 19(3): 1107–1130

    Article  MathSciNet  Google Scholar 

  40. Lu C, Tang J, Yan S, Lin Z. Nonconvex nonsmooth low rank minimization via iteratively reweighted nuclear norm. IEEE Transactions Image Process, 2016, 25(2): 829–839

    Article  MathSciNet  Google Scholar 

  41. Phillips P J, Wechsler H, Huang J, Rauss P J. The feret database and evaluation procedure for face-recognition algorithms. Image and Vision Computing, 1998, 16(5): 295–306

    Article  Google Scholar 

  42. Samaria F S, Harter A C. Parameterisation of a stochastic model for human face identification. In: Proceedings of IEEE Workshop on Applications of Computer Vision. 1994, 138–142

  43. Hollander M, Wolfe D A, Chicken E. Nonparametric Statistical Methods. 3rd ed. New York: Wiley, 1999

    MATH  Google Scholar 

Download references

Acknowledgements

The work described in this paper was partially supported by the National Natural Science Foundation of China (Grant Nos. 61673249, 61703252), the Union Fund of National Natural Science Foundation of China (U1805263), the Research Project Supported by Shanxi Scholarship Council of China (2016-004).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shanxi University, Taiyuan, 030006, China

    Xuejun Wang

  2. School of Science, China Jiliang University, Hangzhou, 310018, China

    Feilong Cao

  3. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Taiyuan, 030006, China

    Wenjian Wang

Authors
  1. Xuejun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feilong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenjian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjian Wang.

Additional information

Xuejun Wang received the BS degree in mathematics and applied mathematics from Agricultural University of Hebei, China in 2011, and the MS degree in applied mathematics from China Jiliang University, China in 2014. He is currently pursuing the PhD degree in college of computer science and technology, Shanxi University, China.

His current research interests include pattern recognition, sparse representation.

Feilong Cao is a professor of Applied Mathematics and Information Computation Science with China Jiliang University. China. He received the BS and MS degrees from Ningxia University, China in 1987 and 1998, respectively, and the PhD degree from Xi’an Jiaotong University, China in 2003, all in allied mathematics. He became a full professor in 2002. He was a research fellow with the Center of Basic Sciences, Xi’an Jiaotong University, China from 2003 to 2004, where he was a Post-Doctoral Research Fellow with the School of Aerospace from 2004 to 2006. He has authored or co-authored two books and over 100 scientific papers in refereed journals.

His current research interests include pattern recognition, neural networks, and approximation theory.

Wenjian Wang is a professor and PhD supervisor of Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, China. She received PhD degree in applied mathematics from Xi’an Jiaotong University, China in 2004. and now serves as a PhD supervisor in Computer Application Technology and System Engineering. She has published more than 100 academic papers on machine learning, computational intelligence, and data mining.

Her current research interests include machine learning, data mining, and artificial intelligence, etc.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Cao, F. & Wang, W. Adaptive sparse and dense hybrid representation with nonconvex optimization. Front. Comput. Sci. 14, 144306 (2020). https://doi.org/10.1007/s11704-019-7200-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-019-7200-y

Keywords

Search

Navigation