Abstract
Recognition of human actions from visual contents is a budding field of computer vision and image understanding. The problem with such a recognition system is the huge dimensions of the feature vectors. Many of these features are irrelevant to the classification mechanism. For this reason, in this paper, we propose a novel feature selection (FS) model called cooperative genetic algorithm (CGA) to select some of the most important and discriminating features from the entire feature set to improve the classification accuracy as well as the time requirement of the activity recognition mechanism. In CGA, we have made an effort to embed the concepts of cooperative game theory in GA to create a both-way reinforcement mechanism to improve the solution of the FS model. The proposed FS model is tested on four benchmark video datasets named Weizmann, KTH, UCF11, HMDB51, and two sensor-based UCI HAR datasets. The experiments are conducted using four state-of-the-art feature descriptors, namely HOG, GLCM, SURF, and GIST. It is found that there is a significant improvement in the overall classification accuracy while considering very small fraction of the original feature vector.
Similar content being viewed by others
References
Aslan MF, Durdu A, Sabanci K (2020) Human action recognition with bag of visual words using different machine learning methods and hyperparameter optimization. Neural Comput Appl 32:8585–8597
Sahoo SP, Ari S (2019) On an algorithm for human action recognition. Expert Syst Appl 115:524–534
Saggese A, Strisciuglio N, Vento M, Petkov N (2019) Learning skeleton representations for human action recognition. Pattern Recognit Lett 118:23–31
Zhang P, Lan C, Xing J et al (2019) View adaptive neural networks for high performance skeleton-based human action recognition. IEEE Trans Pattern Anal Mach Intell 41:1963–1978
Ahmed S, Ghosh KK, Singh PK et al (2020) Hybrid of harmony search algorithm and ring theory-based evolutionary algorithm for feature selection. IEEE Access 8:102629–102645
Ghosh KK, Singh PK, Hong J et al (2020) Binary social mimic optimization algorithm with X-shaped transfer function for feature selection. IEEE Access 8(1):97890–97906. https://doi.org/10.1109/ACCESS.2020.2996611
Ghosh KK, Ahmed S, Singh PK et al (2020) Improved binary sailfish optimizer based on adaptive β-hill climbing for feature selection. IEEE Access 8(1):83548–83560. https://doi.org/10.1109/ACCESS.2020.2991543
Chatterjee B, Bhattacharyya T, Ghosh KK et al (2020) Late Acceptance Hill Climbing Based Social Ski Driver Algorithm for Feature Selection. IEEE Access 8:75393–75408. https://doi.org/10.1109/ACCESS.2020.2988157
Ghosh M, Guha R, Mondal R et al (2018) Feature selection using histogram-based multi-objective GA for handwritten Devanagari numeral recognition. In: Bhateja V, Coello Coello C, Satapathy S, Pattnaik P (eds) Intelligent engineering informatics. Advances in intelligent systems and computing, vol 695. Springer, Singapore, pp 471–479. https://doi.org/10.1007/978-981-10-7566-7_46
Ghosh M, Adhikary S, Ghosh KK et al (2019) Genetic algorithm based cancerous gene identification from microarray data using ensemble of filter methods. Med Biol Eng Comput 57:159–176
Ghosh M, Malakar S, Bhowmik S et al (2019) Feature selection for handwritten word recognition using memetic algorithm. In: Mandal J, Dutta P, Mukhopadhyay S (eds) Advances in intelligent computing. Studies in computational intelligence, vol 687. Springer, Singapore. https://doi.org/10.1007/978-981-10-8974-9_6
Malakar S, Ghosh M, Bhowmik S et al (2020) A GA based hierarchical feature selection approach for handwritten word recognition. Neural Comput Appl 32:2533–2552
Singh PK, Sarkar R, Das N (2018) Benchmark databases of handwritten Bangla - Roman and Devanagari—Roman mixed-script document images. Multimedia Tools Appl 77:8441–8473
Ghosh M, Kundu T, Ghosh D, Sarkar R (2019) Feature selection for facial emotion recognition using late hill-climbing based memetic algorithm. Multimed Tools Appl 78:25753–25779. https://doi.org/10.1007/s11042-019-07811-x
Saha S, Ghosh M, Ghosh S et al (2020) Feature Selection for Facial Emotion Recognition Using Cosine Similarity-Based Harmony Search Algorithm. Appl Sci 10:2816
Liu H, Motoda H (2007) Computational methods of feature selection. CRC Press, Boca Raton
Mitra P, Murthy CA, Pal SK (2002) Unsupervised feature selection using feature similarity. IEEE Trans Pattern Anal Mach Intell 24:301–312
Shang W-Q, Qu Y-L, Huang H-K et al (2006) Fuzzy knn text classifier based on gini index. J Guang xi Norm Univ Nat Sci Ed 24:87–90
Guha R, Ghosh M, Chakrabarti A et al (2020) Introducing clustering based population in Binary Gravitational Search Algorithm for Feature Selection. Appl Soft Comput 93:106341. https://doi.org/10.1016/j.asoc.2020.106341
Dorigo M, Birattari M (2011) Ant colony optimization. In: Sammut C, Webb GI (eds) Encyclopedia of machine learning. Springer, pp 37–40
Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: Micro Machine and Human Science, 1995. MHS’95., Proceedings of the Sixth International Symposium on. IEEE, pp 39–43
Ghosh M, Guha R, Singh PK et al (2019) A histogram based fuzzy ensemble technique for feature selection. Evol Intell 12(4):713–724
Ghosh M, Begum S, Sarkar R et al (2019) Recursive memetic algorithm for gene selection in microarray data. Expert Syst Appl 116:172–185
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. IEEE Comput Soc Conf Comput Vis Pattern Recognit 1:886–893
Haralick RM, Shanmugam K Its’Hak Dinstein (1973) Textural Features for Image Classification. IEEE Trans Syst Man Cybern SMC-3(6):610–621
Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. In: European conference on computer vision. Springer, pp 404–417
Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42:145–175. https://doi.org/10.1023/A:1011139631724
Blank M, Gorelick L, Shechtman E, et al (2005) Actions as space-time shapes. In: Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1. IEEE, pp 1395–1402
Schuldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. IEEE, pp 32–36
Liu J, Luo J, Shah M (2009) Recognizing realistic actions from videos in the wild. In: IEEE conference on computer vision and pattern recognition. Citeseer, pp 1996–2003
Kuehne H, Jhuang H, Garrote E, et al (2011) HMDB: a large video database for human motion recognition. In: 2011 International conference on computer vision. IEEE, pp 2556–2563
Anguita D, Ghio A, Oneto L et al (2013) A public domain dataset for human activity recognition using smartphones. In: Esann
Anguita D, Ghio A, Oneto L, et al (2012) Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: International workshop on ambient assisted living. Springer, pp 216–223
Niebles JC, Fei-Fei L (2007) A hierarchical model of shape and appearance for human action classification. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8
Scovanner P, Ali S, Shah M (2007) A 3-dimensional sift descriptor and its application to action recognition. In: Proceedings of the 15th ACM international conference on Multimedia. pp 357–360
Niebles JC, Wang H, Fei-Fei L (2008) Unsupervised learning of human action categories using spatial-temporal words. Int J Comput Vis 79:299–318
Ikizler-Cinbis N, Sclaroff S (2010) Object, scene and actions: Combining multiple features for human action recognition. In: European conference on computer vision. Springer, pp 494–507
Huang W, Wu QMJ (2010) Human action recognition based on self organizing map. In: 2010 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 2130–2133
Chakraborty B, Holte MB, Moeslund TB, et al (2011) A selective spatio-temporal interest point detector for human action recognition in complex scenes. 1776–1783
Reddy KK, Cuntoor N, Perera A, Hoogs A (2012) Human action recognition in large-scale datasets using histogram of spatiotemporal gradients. In: 2012 IEEE ninth international conference on advanced video and signal-based surveillance. IEEE, pp 106–111
Yan X, Luo Y (2012) Recognizing human actions using a new descriptor based on spatial–temporal interest points and weighted-output classifier. Neurocomputing 87:51–61
Yuan C, Li X, Hu W, et al (2013) 3D R transform on spatio-temporal interest points for action recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp 724–730
Wang L, Qiao Y, Tang X (2013) Motionlets: Mid-level 3D parts for human motion recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2674–2681
Sadek S, Al-Hamadi A, Krell G, Michaelis B (2013) Affine-invariant feature extraction for activity recognition. ISRN Mach Vis. Article ID 215195. https://doi.org/10.1155/2013/215195
Solmaz B, Assari SM, Shah M (2013) Classifying web videos using a global video descriptor. Mach Vis Appl 24:1473–1485
Wang H, Kläser A, Schmid C, Liu C-L (2013) Dense trajectories and motion boundary descriptors for action recognition. Int J Comput Vis 103:60–79
Cai Z, Wang L, Peng X, Qiao Y (2014) Multi-view super vector for action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 596–603
Wu J, Hu D, Chen F (2014) Action recognition by hidden temporal models. Vis Comput 30:1395–1404. https://doi.org/10.1007/s00371-013-0899-9
Zhou W, Zhang Z (2014) Human action recognition with multiple-instance Markov model. IEEE Trans Inf Forensics Secur 9:1581–1591
Seto S, Zhang W, Zhou Y (2015) Multivariate time series classification using dynamic time warping template selection for human activity recognition. In: 2015 IEEE symposium series on computational intelligence. IEEE, pp 1399–1406
Chen CY, Grauman K (2017) Efficient activity detection in untrimmed video with max-subgraph search. IEEE Trans Pattern Anal Mach Intell 39(5):908–921. https://doi.org/10.1109/TPAMI.2016.2564404
Kushwaha AKS, Srivastava RA (2020) Framework for human activity recognition using pose feature for video surveillance system. Int J Comput Appl 975:8887
Luvizon DC, Tabia H, Picard D (2017) Learning features combination for human action recognition from skeleton sequences. Pattern Recognit Lett 99:13–20
Sharif M, Khan MA, Akram T et al (2017) A framework of human detection and action recognition based on uniform segmentation and combination of Euclidean distance and joint entropy-based features selection. Eurasip J Image Video Process. https://doi.org/10.1186/s13640-017-0236-8
Singh R, Kushwaha AKS, Srivastava R (2019) Multi-view recognition system for human activity based on multiple features for video surveillance system. Multimed Tools Appl 78:17165–17196
Sahoo SP, Silambarasi R, Ari S (2019) Fusion of histogram based features for Human Action Recognition. In: 2019 5th international conference on advanced computing and communication systems, ICACCS 2019. IEEE, pp 1012–1016
Gupta S, Ghosh Mazumdar S (2013) Sobel Edge detection algorithm. Int J Comput Sci Manag Res 2:1578–1583
Kolosnjaji B, Eckert C (2015) Neural network-based user-independent physical activity recognition for mobile devices. In: International conference on intelligent data engineering and automated learning. Springer, pp 378–386
Kim Y-J, Kang B-N, Kim D (2015) Hidden Markov model ensemble for activity recognition using tri-axis accelerometer. In: 2015 IEEE international conference on systems, man, and cybernetics. IEEE, pp 3036–3041
Acharjee D, Mukherjee A, Mandal JK, Mukherjee N (2016) Activity recognition system using inbuilt sensors of smart mobile phone and minimizing feature vectors. Microsyst Technol 22:2715–2722
Sousa W, Souto E, Rodrigres J, et al (2017) A comparative analysis of the impact of features on human activity recognition with smartphone sensors. In: Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web. pp 397–404
BhanuJyothi K, Bindu KH, Suryanarayana D (2017) A Comparative Study of Random Forest & K-Nearest Neighbors on HAR dataset Using Caret. IJIRT 3:6–9
Sikder N, Chowdhury MS, Arif AMS, Nahid A-A (2019) Human activity recognition using multichannel convolutional neural network. In: 2019 5th International conference on advanced electrical engineering
Bhattacharya S, Shaw V, Singh PK, et al (2019) SV-NET: A deep learning approach to video based human activity recognition. In: Proceedings of the eleventh international conference on soft computing and pattern recognition, SoCPaR 2019. Hyderabad, India, pp 13–15
Sadhukhan S, Mallick S, Singh PK et al (2020) A comparative study of different feature descriptors for video-based human action recognition. In: Mandal J, Banerjee S (eds) Intelligent computing: image processing based applications. Advances in intelligent systems and computing, vol 1157. Springer, Singapore. https://doi.org/10.1007/978-981-15-4288-6_3
Rosati S, Balestra G, Knaflitz M (2018) Comparison of different sets of features for human activity recognition by wearable sensors. Sensors 18:4189
Zainudin MNS, SULAIMAN MDNBIN, Mustapha N et al (2018) Two-stage feature selection using ranking self-adaptive differential evolution algorithm for recognition of acceleration activity. Turkish J Electr Eng Comput Sci 26:1378–1389
Holland JH (1992) Genetic algorithms. Sci Am 1:66–73
Abualigah LMQ, Hanandeh ES (2015) Applying genetic algorithms to information retrieval using vector space model. Int J Comput Sci Eng Appl 5:19
Nezamabadi-Pour H (2015) A quantum-inspired gravitational search algorithm for binary encoded optimization problems. Eng Appl Artif Intell 40:62–75. https://doi.org/10.1016/j.engappai.2015.01.002
Miller BL, Goldberg DE (1995) Genetic algorithms, tournament selection, and the effects of noise. Complex Syst 9:193–212
Lipowski A, Lipowska D (2012) Roulette-wheel selection via stochastic acceptance. Phys A Stat Mech its Appl 391:2193–2196. https://doi.org/10.1016/j.physa.2011.12.004
Guha R, Ghosh M, Singh PK et al (2019) M-HMOGA: a new multi-objective feature selection algorithm for handwritten numeral classification. J Intell Syst 29:1453–1467. https://doi.org/10.1515/jisys-2019-0064
Guha R, Ghosh M, Kapri S et al (2019) Deluge based genetic algorithm for feature selection. Evol Intell. https://doi.org/10.1007/s12065-019-00218-5
Ghosh M, Bera SK, Guha R, Sarkar R (2019) Contrast enhancement of degraded document image using partitioning based genetic algorithm. In: International conference on emerging technologies for sustainable development (ICETSD’19). pp 431–435
Davis M, Maschler M (1965) The kernel of a cooperative game. Nav Res Logist Q 12:223–259
Bilbao JM (2012) Cooperative games on combinatorial structures. Springer, Berlin
Mukaka MM (2012) A guide to appropriate use of correlation coefficient in medical research. Malawi Med J 24:69–71
Lawrence I, Lin K (1989) A concordance correlation coefficient to evaluate reproducibility. Biometrics 45(1):255–268
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420
Guha R, Ghosh KK, Bhowmik S, Sarkar R (2020) Mutually informed correlation coefficient (MICC)—a new filter based feature selection method. In: 2020 IEEE Calcutta conference (CALCON), Kolkata, India, pp 54–58. https://doi.org/10.1109/CALCON49167.2020.9106516
Estévez PA, Tesmer M, Perez CA, Zurada JM (2009) Normalized mutual information feature selection. IEEE Trans Neural Netw 20:189–201
Amiri F, Yousefi MR, Lucas C et al (2011) Mutual information-based feature selection for intrusion detection systems. J Netw Comput Appl 34:1184–1199
Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90:1151–1163
Ngiam J, Koh PW, Chen Z, Bhaskar SA, Ng AY (2011) Sparse filtering. NIPS 11:1125–1133
Dua, D. and Graff C (2019) UCI Machine Learning Repository. In: Irvine, CA Univ. California, Sch. Inf. Comput. Sci. http://archive.ics.uci.edu/ml. Accessed 7 Jan 2019
Basu S, Das N, Sarkar R − , et al (2005) Handwritten ‘Bangla’ alphabet recognition using an MLP based classfier. In: 2nd National conference on computer processing of Bangla-2005. pp 285–291
Siedlecki W, Sklansky J (1993) A note on genetic algorithms for large-scale feature selection. Handbook of pattern recognition and computer vision. World Scientific, vol 10, pp 88–107
Kennedy J, Eberhart RC (1997) Discrete binary version of the particle swarm algorithm. In: Proceedings of the IEEE international conference on systems, man and cybernetics. pp 4104–4108
Rashedi E, Nezamabadi-Pour H, Saryazdi S (2010) BGSA: binary gravitational search algorithm. Nat Comput 9:727–745. https://doi.org/10.1007/s11047-009-9175-3
Ghosh M, Guha R, Alam I et al (2019) Binary genetic swarm optimization: a combination of GA and PSO for feature selection. J Intell Syst 29:1598–1610
Ghosh M, Guha R, Sarkar R, Abraham A (2019) A wrapper-filter feature selection technique based on ant colony optimization. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04171-3
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interests regarding the publication of this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
1.1 Pearson Correlation Coefficient
Pearson correlation coefficient (PCC) is a correlation measure that is used in statistics to find the strength of linear dependence between two random variables \( x \) and \( y \). It is denoted by the following equation:
where \( r_{xy} \) is the PCC value, \( x_{i} \) and \( y_{i} \) represents \( i{\text{th}} \) sample in \( x \) and \( y, \) respectively, and \( n \) is the total number of samples.
In the present context, we have considered different features as random variables. From the equation, we know that the PCC gives a real value between − 1 to +1 for any two features. A value lesser than 0 means both the features are inversely dependent on each other, i.e., if one variable increases, others will decrease. A value equal to 0 means both the features are independent of each other. A value greater than 0 signifies, they both are directly dependent on each other, i.e., if one feature increases, then the other will also increase.
1.2 Mutual information
Mutual information (MI) is a measure between two (possibly multi-dimensional) random variables \( X \) and \( Y \) that quantifies the amount of information obtained about one random variable, through the other random variable. The mutual information is given by
where \( P_{xy} \left( {x,y} \right) \) is the joint probability density function of \( X \) and \( Y \), and \( P_{x} \left( x \right) \) and \( P_{y} \left( y \right) \) are the marginal density functions. The mutual information determines how similar the joint distribution \( P_{xy} \left( {x,y} \right) \) is to the products of the factored marginal distributions. If X and Y are completely unrelated (and therefore independent), then \( P_{xy} \left( {x,y} \right) \) would equal \( P_{x} \left( x \right)P_{y} \left( y \right) \), and this integral would be zero.
1.3 Cooperative game theory
In the world of game theory, cooperative game (also known as coalition game) is a well-known game that revolves around the formation of groups or coalitions of players depending on collective payoffs. Let there be \( n\left( { > = 2} \right) \) players playing the game. \( N \) represents the entire set of players as \( \{ 1, \, 2, \, 3 \ldots n\} . \) A coalition is defined as a subset \( C \subset N \). So, there are \( 2^{n} \) ways of selecting a coalition. A formal definition of the coalition can be found in [68] as:
The coalitional form of an \( n \)-person game is given by the pair (\( N,f \)), where \( N = \{ 1, \, 2, \ldots ,n\} \) is the set of players and \( f \) is a real-valued function, called the characteristic function of the game, defined on the set, \( 2^{N} \), of all coalitions (subsets of \( N \)), and satisfying
-
i.
\( f(\emptyset ) \, = \, 0, \) where \( \emptyset \) is the empty coalition.
-
ii.
(super-additivity) if \( S \) and T are disjoint coalitions (\( (S \cap T = \emptyset ), \) then \( f(S) + f(T) \le f(S \cup T). \)
The value \( f(C) \) is a real value for any coalition \( C \subset N \). It gives us a measure of strength or worth possessed by the coalition. So, the ultimate purpose of the game becomes to find a worthy coalition of players subject to some application-related constraints.
Rights and permissions
About this article
Cite this article
Guha, R., Khan, A.H., Singh, P.K. et al. CGA: a new feature selection model for visual human action recognition. Neural Comput & Applic 33, 5267–5286 (2021). https://doi.org/10.1007/s00521-020-05297-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05297-5