Skip to main content
Log in

Multi-label classification by formulating label-specific features from simultaneous instance level and feature level

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-label learning (MLL) trains a classification model from multiple labelled datasets, where each training instance is annotated with a set of class labels simultaneously. Following the binary relevance MLL paradigm, a recently effective spirit is to constructing specific features for each label, instead of training over the original feature space. Existing label-specific methods, however, only consider the information from instance distributions, making the reconstructed features poorly discriminative. In this paper, we propose the generation of Label-spEcific feaTures by simultaneously exploring insTance distributions and fEatuRe distributions, and suggest a new method named Letter. Letter reconstructs two subsets of new features from the instance level and feature level, respectively. More concretely, from the instance level, Letter incorporates a sparse constraint, and from the feature level, we cluster the original features to construct new features as an extension. The combination of these two new feature subsets is the final set of label-specific features. Extensive experiments on a total of 14 benchmark datasets verify the competitive performance of Letter against the existing state-of-the-art MLL methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. |⋅| returns the set cardinality.

  2. We employ the Euclidean as the metric in this paper.

  3. http://mulan.sourceforge.net/datasets-mlc.html

  4. http://palm.seu.edu.cn/zhangml/files/ML-kNN.rar

  5. http://palm.seu.edu.cn/zhangml/files/LIFT.rar

  6. https://github.com/guanyy217/Applied-Intelligence_LETTER

References

  1. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771

    Article  Google Scholar 

  2. Cesa-Bianchi N, Re M, Valentini G (2012) Synergy of multi-label hierarchical ensembles, data fusion, and cost-sensitive methods for gene functional inference. Mach Learn 88(1-2):209–241

    Article  MathSciNet  Google Scholar 

  3. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27

    Article  Google Scholar 

  4. Chen Y, Zhang F, Zuo W (2018) Deep image annotation and classification by fusing multi-modal semantic topics. KSII Trans Internet Inf Syst 12(1):392–412

    Google Scholar 

  5. Cheng W, Hüllermeier E (2009) Combining instance-based learning and logistic regression for multilabel classification. Mach Learn 76(2-3):211–225

    Article  Google Scholar 

  6. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  7. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Advances in neural information processing systems, pp 681–687

  8. Fürnkranz J, Hüllermeier E, Mencía EL, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153

    Article  Google Scholar 

  9. Gong T, Liu B, Chu Q, Yu N (2019) Using multi-label classification to improve object detection. Neurocomputing 370:174–185

    Article  Google Scholar 

  10. Guo Y, Chung F, Li G, Wang J, Gee JC (2019) Leveraging label-specific discriminant mapping features for multi-label learning. ACM Trans Knowl Discov Data 13(2):24

    Article  Google Scholar 

  11. He ZF, Yang M (2019) Sparse and low-rank representation for multi-label classification. Appl Intell 49(5):1708–1723

    Article  Google Scholar 

  12. Huang J, Li G, Huang Q, Wu X (2015) Learning label specific features for multi-label classification. In: IEEE International conference on data mining, pp 181–190

  13. Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323

    Article  Google Scholar 

  14. Huang J, Li G, Huang Q, Wu X (2017) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876–889

    Article  Google Scholar 

  15. Jabreel M, Moreno A (2019) A deep learning-based approach for multi-label emotion classification in tweets. Appl Sci 9(6): 1123

    Article  Google Scholar 

  16. Li X, Ouyang J, Zhou X (2015) Centroid prior topic model for multi-label classification. Pattern Recogn Lett 62:8–13

    Article  Google Scholar 

  17. Li X, Ouyang J, Zhou X (2015) Supervised topic models for multi-label classification. Neurocomputing 149:811–819

    Article  Google Scholar 

  18. Li X, Ouyang J, Zhou X (2016) Labelset topic model for multi-label document classification. J Intell Inf Syst 46(4):83–97

    Article  Google Scholar 

  19. Li X, Ouyang J, Zhou X, Lu Y, Liu Y (2015) Supervised labeled latent dirichlet allocation for document categorization. Appl Intell 42(3):581–593

    Article  Google Scholar 

  20. Li X, Ouyang J, Zou Y (2018) Supervised topic models with weighted words: multi-label document classification. Front Inf Technol Electron Eng 19(4):513–523

    Article  Google Scholar 

  21. Ma J, Zhang H, Chow TW (2019) Multilabel classification with label-specific features and classifiers: a coarse-and fine-tuned framework. IEEE Trans Cybern:1–15

  22. Read J, Pfahringer B, Holmes G, Frank E (2009) Classifier chains for multi-label classification. In: Machine learning and knowledge discovery in databases, pp 254–269

  23. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333

    Article  MathSciNet  Google Scholar 

  24. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  25. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas I (2008) Multi-label classification of music into emotions. In: International society for music information retrieval, vol 8, pp 325–330

  26. Tsoumakas G, Katakis I, Vlahavas I (2009) Mining multi-label data. In: Data mining and knowledge discovery handbook, pp 667–685

  27. Wang T, Liu L, Liu N, Zhang H, Zhang L, Feng S (2020) A multi-label text classification method via dynamic semantic representation model and deep neural network. Appl Intell:1–13

  28. Wang Y, Rao Y, Zhan X, Chen H, Luo M, Yin J (2016) Sentiment and emotion classification over noisy labels. Knowl-Based Syst 111:207–216

    Article  Google Scholar 

  29. Wei X, Yu Z, Zhang C, Hu Q (2018) Ensemble of label specific features for multi-label classification. In: IEEE International conference on multimedia and expo, pp 1–6

  30. Weng W, Chen YN, Chen CL, Wu SX, Liu JH (2020) Non-sparse label specific features selection for multi-label classification. Neurocomputing 377:85–94

    Article  Google Scholar 

  31. Xu S, Yang X, Yu H, Yu DJ, Yang J, Tsang EC (2016) Multi-label learning with label-specific feature reduction. Knowl-Based Syst 104:52–61

    Article  Google Scholar 

  32. Yu Q, Wang J, Zhang S, Gong Y, Zhao J (2017) Combining local and global hypotheses in deep neural network for multi-label image classification. Neurocomputing 235:38–45

    Article  Google Scholar 

  33. Zhang J, Li C, Cao D, Lin Y, Su S, Dai L, Li S (2018) Multi-label learning with label-specific features by resolving label correlations. Knowl-Based Syst 159:148–157

    Article  Google Scholar 

  34. Zhang JJ, Fang M, Li X (2015) Multi-label learning with discriminative features for each label. Neurocomputing 154:305–316

    Article  Google Scholar 

  35. Zhang ML (2011) Lift: Multi-label learning with label-specific features. In: International joint conference on artificial intelligence, pp 1609–1614

  36. Zhang ML, Li YK, Liu XY, Geng X (2018) Binary relevance for multi-label learning: an overview. Front Comput Sci 12(2):191–202

    Article  Google Scholar 

  37. Zhang ML, Wu L (2015) Lift: Multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(11):107–120

    Article  Google Scholar 

  38. Zhang ML, Zhou ZH (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048

    Article  Google Scholar 

  39. Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  40. Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 48(12):4615–4625

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (NSFC) No.51805203, the Science and Technology Development Plan of Jilin Province 20190201023JC, and the Development and Reform Commission of Jilin Province 2019C054-2.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wenhui Li.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guan, Y., Li, W., Zhang, B. et al. Multi-label classification by formulating label-specific features from simultaneous instance level and feature level. Appl Intell 51, 3375–3390 (2021). https://doi.org/10.1007/s10489-020-02008-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02008-4

Keywords

Navigation