Abstract
Multi-label feature selection is a critical dimension reduction technique in multi-label learning. In conventional multi-label feature selection methods based on information theory, feature relevance is evaluated by mutual information between candidate features and each label. However, previous methods ignore two issues: the influence of the already-selected features on the feature relevance and the influence of the correlations among labels on the feature relevance. To address these two issues, we design a new feature relevance term named Double Conditional Relevance (DCR) that employs two conditional mutual information terms to take the already-selected features and the correlations among labels into account. Finally, a novel multi-label feature selection method combining the new feature relevance term with feature redundancy term is proposed, the proposed method is named Double Conditional Relevance-Multi-label Feature Selection (DCR-MFS). Additionally, an extended method DCR-MFSarg is designed to avoid the shortcoming of the inconsistency of magnitude in DCR-MFS method. The experimental results together with theoretical analysis demonstrate the superiority of the proposed methods.
Similar content being viewed by others
References
Bostani H, Sheikhan M (2017) Hybrid of binary gravitational search algorithm and mutual information for feature selection in intrusion detection systems. Soft Comput 21(9):2307–2324
Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771
Cover TM, Thomas JA (2012) Elements of information theory. Wiley, New York
Doquire G, Verleysen M (2011) Feature selection for multi-label classification problems. In: International work-conference on artificial neural networks. Springer, pp 9–16
Doquire G, Verleysen M (2013) Mutual information-based feature selection for multilabel classification. Neurocomputing 122:148–155
Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Advances in neural information processing systems, pp 681–687
Gao W, Hu L, Zhang P (2018) Class-specific mutual information variation for feature selection. Pattern Recogn 79:328–339
Gao W, Hu L, Zhang P, Wang F (2018) Feature selection by integrating two groups of feature evaluation criteria. Expert Syst Appl 110:11–19
Gonzalezlopez J, Ventura S, Cano A (2020) Distributed multi-label feature selection using individual mutual information measures. Knowl Based Syst 188(105):052
Hancer E (2018) Differential evolution for feature selection: A fuzzy wrapper–filter approach. Soft Comput 1–16
Hu L, Gao W, Zhao K, Zhang P, Wang F (2018) Feature selection considering two types of feature relevancy and feature interdependency. Expert Syst Appl 93:423–434
Hu L, Li Y, Gao W, Zhang P, Hu J (2020) Multi-label feature selection with shared common mode. Pattern Recognit 107344
Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. In: IJCAI, pp 1627–1633
Lee J, Kim DW (2013) Feature selection for multi-label classification using multivariate mutual information. Pattern Recogn Lett 34(3):349–357
Lee J, Kim DW (2015) Fast multi-label feature selection based on information-theoretic feature ranking. Pattern Recogn 48(9):2761–2771
Lee J, Kim DW (2015) Memetic feature selection algorithm for multi-label classification. Inf Sci 293:80–96
Lee J, Kim DW (2015) Mutual information-based multi-label feature selection using interaction information. Expert Syst Appl 42(4):2013–2025
Lee J, Kim DW (2017) Scls: Multi-label feature selection based on scalable criterion for large label set. Pattern Recogn 66:342–352
Li F, Miao D, Pedrycz W (2017) Granular multi-label feature selection based on mutual information. Pattern Recogn 67:410–423
Lin Y, Hu Q, Liu J, Chen J, Duan J (2016) Multi-label feature selection based on neighborhood mutual information. Appl Soft Comput 38:244–256
Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168(C):92–103
Lin Y, Hu Q, Liu J, Li J, Wu X (2017) Streaming feature selection for multilabel learning based on fuzzy mutual information. IEEE Trans Fuzzy Syst 25(6):1491–1507
Lin Y, Hu Q, Zhang J, Wu X (2016) Multi-label feature selection with streaming labels. Inf Sci 372:256–275
Masood MK, Soh YC, Jiang C (2017) Occupancy estimation from environmental parameters using wrapper and hybrid feature selection. Appl Soft Comput 60:482–494
Monard MC, Tsoumakas G, Lee HD (2016) A systematic review of multi-label feature selection and a new method based on label construction. Neurocomputing 180(C):3–15
Oreski D, Oreski S, Klicek B (2017) Effects of dataset characteristics on the performance of feature selection techniques. Appl Soft Comput 52:109–119
Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238
Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings 2008 New Zealand computer science research student conference (NZCSRS 2008), vol 143150
Read J, Martino L, Hollmén J (2017) Multi-label methods for prediction with sequential data. Pattern Recogn 63:45–55
Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Machine Learn 39(2-3):135–168
Shannon CE (2001) A mathematical theory of communication. ACM SIGMOBILE Mob Comput Commun Rev 5(1):3–55
Song XF, Zhang Y, Guo YN, Sun XY, Wang YL (2020) Variable-size cooperative coevolutionary particle swarm optimization for feature selection on high-dimensional data. IEEE Trans Evol Comput 24(5):882–895
SpolaôR N, Cherman EA, Monard MC, Lee HD (2013) A comparison of multi-label feature selection methods using the problem transformation approach. Electron Notes Theoret Comput Sci 292:135–151
Swami A, Jain R (2013) Scikit-learn: Machine learning in python. J Mach Learn Res 12 (10):2825–2830
Szymański P, Kajdanowicz T (2017) A scikit-based python environment for performing multi-label classification. arXiv:1702.01460
Trohidis K, Tsoumakas G, Kalliris G, Vlahavas I (2011) Multi-label classification of music by emotion. EURASIP J Audio Speech Music Process 2011(1):4
Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multi-label classification of music into emotions. In: ISMIR, vol 8, pp 325–330
Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(7):2411–2414
Xu S, Yang X, Yu H, Yu DJ, Yang J, Tsang EC (2016) Multi-label learning with label-specific feature reduction. Knowl-Based Syst 104:52–61
Yu Y, Wang Y (2014) Feature selection for multi-label learning using mutual information and ga. In: International conference on rough sets and knowledge technology. Springer, pp 454– 463
Zhang ML, Peña JM, Robles V (2009) Feature selection for multi-label naive bayes classification. Inf Sci 179(19):3218– 3229
Zhang ML, Zhou ZH (2007) Ml-knn: A lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048
Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 1–11
Zhang Y, Li HG, Wang Q, Peng C (2019) A filter-based bare-bone particle swarm optimization algorithm for unsupervised feature selection. Appl Intell 49:2889–2898
Zhang Y, Wang Q, Gong DW, Song XF (2019) Nonnegative laplacian embedding guided subspace learning for unsupervised feature selection. Pattern Recognit 93:337–352
Acknowledgments
This study was funded by the Postdoctoral Innovative Talents Support Program under Grant No. BX20190137, China Postdoctoral Science Foundation under Grant No. 2020M670839 and National Nature Science Foundation of China (grant number 61772226, 61373051, 61502343); Science and Technology Development Program of Jilin Province (grant number 20140204004GX); Science Research Funds for the Guangxi Universities (grant number KY2015ZD122); Science Research Funds for the Wuzhou University (grant number 2014A002); Project of Science and Technology Innovation Platform of Computing and Software Science (985 Engineering); Key Laboratory for Symbol Computation and Knowledge Engineering of the National Education Ministry of China; Fundamental Research Funds for the Central.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, P., Gao, W. Feature relevance term variation for multi-label feature selection. Appl Intell 51, 5095–5110 (2021). https://doi.org/10.1007/s10489-020-02129-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-02129-w