Skip to main content
Log in

Feature relevance term variation for multi-label feature selection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-label feature selection is a critical dimension reduction technique in multi-label learning. In conventional multi-label feature selection methods based on information theory, feature relevance is evaluated by mutual information between candidate features and each label. However, previous methods ignore two issues: the influence of the already-selected features on the feature relevance and the influence of the correlations among labels on the feature relevance. To address these two issues, we design a new feature relevance term named Double Conditional Relevance (DCR) that employs two conditional mutual information terms to take the already-selected features and the correlations among labels into account. Finally, a novel multi-label feature selection method combining the new feature relevance term with feature redundancy term is proposed, the proposed method is named Double Conditional Relevance-Multi-label Feature Selection (DCR-MFS). Additionally, an extended method DCR-MFSarg is designed to avoid the shortcoming of the inconsistency of magnitude in DCR-MFS method. The experimental results together with theoretical analysis demonstrate the superiority of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bostani H, Sheikhan M (2017) Hybrid of binary gravitational search algorithm and mutual information for feature selection in intrusion detection systems. Soft Comput 21(9):2307–2324

    Article  Google Scholar 

  2. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771

    Article  Google Scholar 

  3. Cover TM, Thomas JA (2012) Elements of information theory. Wiley, New York

    MATH  Google Scholar 

  4. Doquire G, Verleysen M (2011) Feature selection for multi-label classification problems. In: International work-conference on artificial neural networks. Springer, pp 9–16

  5. Doquire G, Verleysen M (2013) Mutual information-based feature selection for multilabel classification. Neurocomputing 122:148–155

    Article  Google Scholar 

  6. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Advances in neural information processing systems, pp 681–687

  7. Gao W, Hu L, Zhang P (2018) Class-specific mutual information variation for feature selection. Pattern Recogn 79:328–339

    Article  Google Scholar 

  8. Gao W, Hu L, Zhang P, Wang F (2018) Feature selection by integrating two groups of feature evaluation criteria. Expert Syst Appl 110:11–19

    Article  Google Scholar 

  9. Gonzalezlopez J, Ventura S, Cano A (2020) Distributed multi-label feature selection using individual mutual information measures. Knowl Based Syst 188(105):052

    Google Scholar 

  10. Hancer E (2018) Differential evolution for feature selection: A fuzzy wrapper–filter approach. Soft Comput 1–16

  11. Hu L, Gao W, Zhao K, Zhang P, Wang F (2018) Feature selection considering two types of feature relevancy and feature interdependency. Expert Syst Appl 93:423–434

    Article  Google Scholar 

  12. Hu L, Li Y, Gao W, Zhang P, Hu J (2020) Multi-label feature selection with shared common mode. Pattern Recognit 107344

  13. Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. In: IJCAI, pp 1627–1633

  14. Lee J, Kim DW (2013) Feature selection for multi-label classification using multivariate mutual information. Pattern Recogn Lett 34(3):349–357

    Article  Google Scholar 

  15. Lee J, Kim DW (2015) Fast multi-label feature selection based on information-theoretic feature ranking. Pattern Recogn 48(9):2761–2771

    Article  Google Scholar 

  16. Lee J, Kim DW (2015) Memetic feature selection algorithm for multi-label classification. Inf Sci 293:80–96

    Article  Google Scholar 

  17. Lee J, Kim DW (2015) Mutual information-based multi-label feature selection using interaction information. Expert Syst Appl 42(4):2013–2025

    Article  Google Scholar 

  18. Lee J, Kim DW (2017) Scls: Multi-label feature selection based on scalable criterion for large label set. Pattern Recogn 66:342–352

    Article  MathSciNet  Google Scholar 

  19. Li F, Miao D, Pedrycz W (2017) Granular multi-label feature selection based on mutual information. Pattern Recogn 67:410–423

    Article  Google Scholar 

  20. Lin Y, Hu Q, Liu J, Chen J, Duan J (2016) Multi-label feature selection based on neighborhood mutual information. Appl Soft Comput 38:244–256

    Article  Google Scholar 

  21. Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168(C):92–103

    Article  Google Scholar 

  22. Lin Y, Hu Q, Liu J, Li J, Wu X (2017) Streaming feature selection for multilabel learning based on fuzzy mutual information. IEEE Trans Fuzzy Syst 25(6):1491–1507

    Article  Google Scholar 

  23. Lin Y, Hu Q, Zhang J, Wu X (2016) Multi-label feature selection with streaming labels. Inf Sci 372:256–275

    Article  Google Scholar 

  24. Masood MK, Soh YC, Jiang C (2017) Occupancy estimation from environmental parameters using wrapper and hybrid feature selection. Appl Soft Comput 60:482–494

    Article  Google Scholar 

  25. Monard MC, Tsoumakas G, Lee HD (2016) A systematic review of multi-label feature selection and a new method based on label construction. Neurocomputing 180(C):3–15

    Google Scholar 

  26. Oreski D, Oreski S, Klicek B (2017) Effects of dataset characteristics on the performance of feature selection techniques. Appl Soft Comput 52:109–119

    Article  Google Scholar 

  27. Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Article  Google Scholar 

  28. Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings 2008 New Zealand computer science research student conference (NZCSRS 2008), vol 143150

  29. Read J, Martino L, Hollmén J (2017) Multi-label methods for prediction with sequential data. Pattern Recogn 63:45–55

    Article  Google Scholar 

  30. Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Machine Learn 39(2-3):135–168

    Article  Google Scholar 

  31. Shannon CE (2001) A mathematical theory of communication. ACM SIGMOBILE Mob Comput Commun Rev 5(1):3–55

    Article  MathSciNet  Google Scholar 

  32. Song XF, Zhang Y, Guo YN, Sun XY, Wang YL (2020) Variable-size cooperative coevolutionary particle swarm optimization for feature selection on high-dimensional data. IEEE Trans Evol Comput 24(5):882–895

    Article  Google Scholar 

  33. SpolaôR N, Cherman EA, Monard MC, Lee HD (2013) A comparison of multi-label feature selection methods using the problem transformation approach. Electron Notes Theoret Comput Sci 292:135–151

    Article  Google Scholar 

  34. Swami A, Jain R (2013) Scikit-learn: Machine learning in python. J Mach Learn Res 12 (10):2825–2830

    MathSciNet  Google Scholar 

  35. Szymański P, Kajdanowicz T (2017) A scikit-based python environment for performing multi-label classification. arXiv:1702.01460

  36. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas I (2011) Multi-label classification of music by emotion. EURASIP J Audio Speech Music Process 2011(1):4

    Article  Google Scholar 

  37. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multi-label classification of music into emotions. In: ISMIR, vol 8, pp 325–330

  38. Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(7):2411–2414

    MathSciNet  MATH  Google Scholar 

  39. Xu S, Yang X, Yu H, Yu DJ, Yang J, Tsang EC (2016) Multi-label learning with label-specific feature reduction. Knowl-Based Syst 104:52–61

    Article  Google Scholar 

  40. Yu Y, Wang Y (2014) Feature selection for multi-label learning using mutual information and ga. In: International conference on rough sets and knowledge technology. Springer, pp 454– 463

  41. Zhang ML, Peña JM, Robles V (2009) Feature selection for multi-label naive bayes classification. Inf Sci 179(19):3218– 3229

    Article  Google Scholar 

  42. Zhang ML, Zhou ZH (2007) Ml-knn: A lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048

    Article  Google Scholar 

  43. Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 1–11

  44. Zhang Y, Li HG, Wang Q, Peng C (2019) A filter-based bare-bone particle swarm optimization algorithm for unsupervised feature selection. Appl Intell 49:2889–2898

    Article  Google Scholar 

  45. Zhang Y, Wang Q, Gong DW, Song XF (2019) Nonnegative laplacian embedding guided subspace learning for unsupervised feature selection. Pattern Recognit 93:337–352

    Article  Google Scholar 

Download references

Acknowledgments

This study was funded by the Postdoctoral Innovative Talents Support Program under Grant No. BX20190137, China Postdoctoral Science Foundation under Grant No. 2020M670839 and National Nature Science Foundation of China (grant number 61772226, 61373051, 61502343); Science and Technology Development Program of Jilin Province (grant number 20140204004GX); Science Research Funds for the Guangxi Universities (grant number KY2015ZD122); Science Research Funds for the Wuzhou University (grant number 2014A002); Project of Science and Technology Innovation Platform of Computing and Software Science (985 Engineering); Key Laboratory for Symbol Computation and Knowledge Engineering of the National Education Ministry of China; Fundamental Research Funds for the Central.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wanfu Gao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, P., Gao, W. Feature relevance term variation for multi-label feature selection. Appl Intell 51, 5095–5110 (2021). https://doi.org/10.1007/s10489-020-02129-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02129-w

Keywords

Navigation