Skip to main content
SpringerLink
Log in

Kernel nonnegative representation-based classifier

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Non-negativity is a critical and explainable property in linear representation-based methods leading to promising performances in the pattern classification field. Based on the non-negativity, a powerful linear representation-based classifier was proposed, namely non-negative representation-based classifier (NRC). With the non-negativity constraint, the NRC enhances the power of the homogeneous samples in the linear representation, while suppressing the representation of the heterogeneous samples, since the homogeneous samples tend to have a positive correlation with the test sample. However, the NRC performs the non-negative representation on the original feature space instead of the high-dimensional non-linear feature space, where it is usually considered when the data samples are not separable with each other. This leads to the poor performance of NRC, especially on high-dimensional data like images. In this paper, we proposed a Kernel Non-negative Representation-based Classifier (KNRC) for addressing this problem to achieve better results in pattern classification. Furthermore, we extended the KNRC to a dimensionality reduction version to reduce the dimensions of the KNRC’s feature space as well as improve its classification ability. We provide extensive numerical experiments including analysis and comparisons on 12 datasets (8 UCI datasets and 4 image datasets) to validate the state-of-the-art performance obtained by the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Akhtar N, Shafait F, Mian A (2017) Efficient classification with sparsity augmented collaborative representation. Pattern Recogn 65:136–145

    Article  Google Scholar 

  2. Anava O, Levy KY (2016) k*-nearest neighbors: From global to local. In: Proceedings of the 30th international conference on neural information processing systems, pp 4923–4931

  3. Bengio Y, Delalleau O, Le Roux N (2005) The curse of dimensionality for local kernel machines. Techn Rep 1258:12

    Google Scholar 

  4. Boyd S, Boyd SP, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge

    Book  Google Scholar 

  5. Boyd S, Parikh N, Chu E (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc

  6. Cai S, Zhang L, Zuo W, Feng X (2016) A probabilistic collaborative representation based approach for pattern classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2950–2959

  7. Chen L, Man H, Nefian AV (2005) Face recognition based on multi-class mapping of fisher scores. Pattern Recogn 38(6):799–811

    Article  Google Scholar 

  8. Cortes C, Vapnik V (1995) Support-vector networks. Machine Learn 20(3):273–297

    MATH  Google Scholar 

  9. Du L, Zhou P, Shi L, Wang H, Fan M, Wang W, Shen YD (2015) Robust multiple kernel k-means using l21-norm. In: Twenty-fourth international joint conference on artificial intelligence

  10. Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  11. Duda RO, Hart PE, Stork DG (2012) Pattern classification. John Wiley & Sons, New York

    MATH  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  13. Hosseini B, Hammer B (2018) Confident kernel sparse coding and dictionary learning. In: 2018 IEEE International conference on data mining (ICDM), pp 1031–1036. IEEE

  14. Hosseini B, Hülsmann F., Botsch M, Hammer B (2016) Non-negative kernel sparse coding for the classification of motion data. In: International conference on artificial neural networks. European Neural Network Society

  15. Hoyer PO (2002) Non-negative sparse coding. In: Proceedings of the 12th IEEE workshop on neural networks for signal processing, pp 557–565. IEEE

  16. Hussain SF (2019) A novel robust kernel for classifying high-dimensional data using support vector machines. Expert Syst Appl 131:116–131

    Article  Google Scholar 

  17. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  Google Scholar 

  18. Liang J, Qin Z, Xiao S, Ou L, Lin X (2019) Efficient and secure decision tree classification for cloud-assisted online diagnosis services. IEEE Transactions on Dependable and Secure Computing

  19. Liu Y, Wei Y, Wang C (2019) Subcortical brain segmentation based on atlas registration and linearized kernel sparse representative classifier. IEEE Access 7:31547–31557

    Article  Google Scholar 

  20. Mercer J (1909) Xvi. functions of positive and negative type, and their connection the theory of integral equations. Philosophical transactions of the royal society of London. Series A, Containing Papers of a Mathematical or Physical Character 209(441-458):415–446

    MATH  Google Scholar 

  21. Minaee S, Boykov YY, Porikli F, Plaza AJ, Kehtarnavaz N, Terzopoulos D (2021) Image segmentation using deep learning: A survey IEEE transactions on pattern analysis and machine intelligence

  22. Nene SA, Nayar SK, Murase H, et al. (1996) Columbia object image library (coil-100)

  23. Ren M, Zeng W, Yang B, Urtasun R (2018) Learning to reweight examples for robust deep learning. In: International conference on machine learning, pp 4334–4343. PMLR

  24. Shawe-Taylor J, Cristianini N, et al. (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Book  Google Scholar 

  25. Shen T, Ott M, Auli M, Ranzato M (2019) Mixture models for diverse machine translation: Tricks of the trade. In: International conference on machine learning, pp 5719–5728. PMLR

  26. Śmieja M, Struski Ł, Tabor J, Marzec M (2019) Generalized rbf kernel for incomplete data. Knowl Based Syst 173:150–162

    Article  Google Scholar 

  27. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  28. Theodoridis S, Koutroumbas K (2009) Chapter 4 - nonlinear classifiers. In: Pattern Recognition (Fourth Edition). fourth edition edn. https://doi.org/10.1016/B978-1-59749-272-0.50006-2. Academic Press, Boston, pp 151–260

  29. Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913

    Article  Google Scholar 

  30. Timofte R, Van Gool L (2014) Adaptive and weighted collaborative representations for image classification. Pattern Recogn Lett 43:127–135

    Article  Google Scholar 

  31. Varikuti DP, Genon S, Sotiras A, Schwender H, Hoffstaedter F, Patil KR, Jockwitz C, Caspers S, Moebus S, Amunts K et al (2018) Evaluation of non-negative matrix factorization of grey matter in age prediction. Neuroimage 173:394–410

    Article  Google Scholar 

  32. Vert JP, Tsuda K, Schölkopf B (2004) A primer on kernel methods. Kernel Methods Comput Biol 47:35–70

    Google Scholar 

  33. Vinge R, McKelvey T (2019) Understanding support vector machines with polynomial kernels. In: 2019 27th European signal processing conference (EUSIPCO), pp 1–5. IEEE

  34. Wang N, Wang J, Yeung DY (2013) Online robust non-negative dictionary learning for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 657–664

  35. Wright J, Ma Y, Mairal J, Sapiro G, Huang TS, Yan S (2010) Sparse representation for computer vision and pattern recognition. Proc IEEE 98(6):1031–1044

    Article  Google Scholar 

  36. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2008) Robust face recognition via sparse representation. IEEE Trans Patt Analy Mach Intell 31(2):210–227

    Article  Google Scholar 

  37. Wu SG, Bao FS, Xu EY, Wang YX, Chang YF, Xiang QL (2007) A leaf recognition algorithm for plant classification using probabilistic neural network. In: 2007 IEEE international symposium on signal processing and information technology, pp 11–16. IEEE

  38. Xie Y, Liu J, Qu Y, Tao D, Zhang W, Dai L, Ma L (2020) Robust kernelized multiview self-representation for subspace clustering. IEEE transactions on neural networks and learning systems

  39. Xu J, An W, Zhang L, Zhang D (2019) Sparse, collaborative, or nonnegative representation: which helps pattern classification? Pattern Recogn 88:679–688

    Article  Google Scholar 

  40. Xu Y, Zhu Q, Fan Z, Zhang D, Mi J, Lai Z (2013) Using the idea of the sparse representation to perform coarse-to-fine face recognition. Inf Sci 238:138–148

    Article  MathSciNet  Google Scholar 

  41. Zeng S, Yang X, Gou J (2017) Multiplication fusion of sparse and collaborative representation for robust face recognition. Multimed Tools Appl 76(20):20889–20907

    Article  Google Scholar 

  42. Zeng S, Zhang B, Gou J, Xu Y (2020) Regularization on augmented data to diversify sparse representation for robust image classification. IEEE Transactions on Cybernetics

  43. Zeng S, Zhang B, Zhang Y, Gou J (2018) Collaboratively weighting deep and classic representation via l_2 regularization for image classification. In: Asian conference on machine learning, pp 502–517. PMLR

  44. Zeng S, Zhang B, Zhang Y, Gou J (2020) Dual sparse learning via data augmentation for robust facial image classification. Int J Mach Learn Cybern 1717–1734

  45. Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: Which helps face recognition?. In: 2011 International conference on computer vision, pp 471–478. IEEE

  46. Zhang L, Zhou WD, Chang PC, Liu J, Yan Z, Wang T, Li FZ (2011) Kernel sparse representation-based classifier. IEEE Trans Signal Process 60(4):1684–1695

    Article  MathSciNet  Google Scholar 

  47. Zhang Y, Xu T, Ma J (2017) Image categorization using non-negative kernel sparse representation. Neurocomputing 269:21–28

    Article  Google Scholar 

  48. Zhou J, Zeng S, Zhang B (2020) Linear representation-based methods for image classification: A survey. IEEE Access 8:216645–216670

    Article  Google Scholar 

  49. Zhou J, Zeng S, Zhang B (2020) Two-stage knowledge transfer framework for image classification. Pattern Recogn 107:107529

    Article  Google Scholar 

  50. Zhu R, Dong M, Xue JH (2019) Learning distance to subspace for the nearest subspace methods in high-dimensional data classification. Inform Sci 481:69–80

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This document is the result of a research project funded by the University of Macau (File no. MYRG2019-00006-FST).

Author information

Authors and Affiliations

  1. PAMI Research Group, Department of Computer and Information Science, Faculty of Science and Technology, University of Macau, Taipa, Macau, China

    Jianhang Zhou, Shaoning Zeng & Bob Zhang

Authors
  1. Jianhang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaoning Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bob Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bob Zhang.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Zeng, S. & Zhang, B. Kernel nonnegative representation-based classifier. Appl Intell 52, 2269–2289 (2022). https://doi.org/10.1007/s10489-021-02486-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02486-0

Keywords

Search

Navigation