Skip to main content
SpringerLink
Log in

J-LDFR: joint low-level and deep neural network feature representations for pedestrian gender classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Appearance-based gender classification is one of the key areas in pedestrian analysis, and it has many useful applications such as visual surveillance, predict demographics statistics, population prediction, and human–computer interaction. For pedestrian gender classification, traditional and deep convolutional neural network (CNN) approaches are employed individually. However, they are facing issues, for instance, discriminative feature representations, lower classification accuracy, and small sample size for model learning. To address these issues, this article proposes a framework that considers the combination of both traditional and deep CNN approaches for gender classification. To realize it, HOG- and LOMO-assisted low-level features are extracted to handle rotation, viewpoint and illumination variances in the images. Simultaneously, VGG19- and ResNet101-based standard deep CNN architectures are employed to acquire the deep features which are robust against pose variations. To avoid the ambiguous and unnecessary feature representations, the entropy-controlled features are picked from both low-level and deep representations of features that reduce the dimension of computed features. By merging the selected low-level features with deep features, we obtain a robust joint feature representation. The extensive experiments are conducted on PETA and MIT datasets, and computed results suggest that using the integration of both low-level and deep feature representations can improve the performance as compared to using these feature representations, individually. The proposed framework achieves AU-ROC of 96% and accuracy of 89.3% on the PETA dataset, and AU-ROC of 86% and accuracy of 82% on the MIT dataset. The experimental outcomes show that the proposed J-LDFR framework outperformed the existing gender classification methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Pławiak P, Abdar M, Acharya UR (2019) Application of new deep genetic cascade ensemble of SVM classifiers to predict the Australian credit scoring. Appl Soft Comput 84:105740

    Google Scholar 

  2. Pławiak P, Abdar M, Pławiak J, Makarenkov V, Acharya UR (2020) DGHNL: a new deep genetic hierarchical network of learners for prediction of credit scoring. Inf Sci 516:401–418

    Google Scholar 

  3. Fathima A, Vaidehi K (2020) Review on facial expression recognition system using machine learning techniques. In: Advances in decision sciences, image Pprocessing, security and computer vision. Springer, Cham, pp 608–618

  4. Pandey P, Pallavi S, Pandey SC (2020) Pragmatic medical image analysis and deep learning: an emerging trend. In: Advancement of machine intelligence in interactive medical image analysis. Springer, Singapore, pp 1–18

    Google Scholar 

  5. Pławiak P, Acharya UR (2019) Novel deep genetic ensemble of classifiers for arrhythmia detection using ECG signals. In: MacIntyre J (ed) Neural computing and applications. Springer, Berlin, pp 1–25

    Google Scholar 

  6. Tuncer T, Dogan S, Pławiak P, Acharya UR (2019) Automated arrhythmia detection using novel hexadecimal local pattern and multilevel wavelet transform with ECG signals. Knowl-Based Syst 186:104923

    Google Scholar 

  7. Cai Z, Saberian MJ, Vasconcelos N (2019) Learning complexity-aware cascades for pedestrian detection. IEEE Trans Pattern Anal Mach Intell

  8. Khan MA, Akram T, Sharif M, Javed MY, Muhammad N, Yasmin M (2018) An implementation of optimized framework for action classification using multilayers neural network on selected fused features. Pattern Anal Appl 22(4):1377–1397

    MathSciNet  Google Scholar 

  9. Yao H, Zhang S, Hong R, Zhang Y, Xu C, Tian Q (2019) Deep representation learning with part loss for person re-identification. IEEE Trans Image Process 28(6):2860–2871

    MathSciNet  MATH  Google Scholar 

  10. Cai L, Zeng H, Zhu J, Cao J, Hou J, Cai C (2017) Multi-view joint learning network for pedestrian gender classification. In: International symposium on intelligent signal processing and communication systems (ISPACS), pp 23–27

  11. Sivabalakrishnan M, Menaka R, Jeeva S (2019) Smart video surveillance systems and identification of human behavior analysis. In: Countering cyber attacks and preserving the integrity and availability of critical systems. IGI Global, pp 64–97

  12. Chen Y, Duffner S, Stoian A, Dufour J-Y, Baskurt A (2018) Deep and low-level feature based attribute learning for person re-identification. Image Vis Comput 79:25–34

    Google Scholar 

  13. Sun Y, Zhang M, Sun Z, Tan T (2017) Demographic analysis from biometric data: achievements, challenges, and new frontiers. IEEE Trans Pattern Anal Mach Intell 40:332–351

    Google Scholar 

  14. Azzopardi G, Greco A, Saggese A, Vento M (2018) Fusion of domain-specific and trainable features for gender recognition from face images. IEEE access 6:24171–24183

    Google Scholar 

  15. Mane S, Shah G (2019) Facial recognition, expression recognition, and gender identification. In: Data management, analytics and innovation. Springer, Singapore, pp 275–290

    Google Scholar 

  16. Cheng J, Li Y, Wang J, Yu L, Wang S (2019) Exploiting effective facial patches for robust gender recognition. Tsinghua Sci Technol 24:333–345

    Google Scholar 

  17. Geetha A, Sundaram M, Vijayakumari B (2019) Gender classification from face images by mixing the classifier outcome of prime, distinct descriptors. Soft Comput 23:2525–2535

    Google Scholar 

  18. Guo G, Mu G, Fu Y (2009) Gender from body: a biologically-inspired approach with manifold learning. In: Asian conference on computer vision, pp 236–245

  19. Ng C-B, Tay Y-H, Goi B-M (2013) A convolutional neural network for pedestrian gender recognition. In: International symposium on neural networks, pp 558–564

  20. Ahmad K, Sohail A, Conci N, De Natale F (2018) A comparative study of global and deep features for the analysis of user-generated natural disaster related images. In: IEEE 13th image, video, and multidimensional signal processing workshop (IVMSP), pp 1–5

  21. Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2197–2206

  22. Afifi M, Abdelhamed A (2019) AFIF4: deep gender classification based on adaboost-based fusion of isolated facial features and foggy faces. J Vis Commun Image Represent 62:77–86

    Google Scholar 

  23. Ng C-B, Tay Y-H, Goi B-M (2015) A review of facial gender recognition. Pattern Anal Appl 18:739–755

    MathSciNet  Google Scholar 

  24. BenAbdelkader C, Griffin P (2005) A local region-based approach to gender classi. cation from face images. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05)-workshops, pp 52–52

  25. Eidinger E, Enbar R, Hassner T (2014) Age and gender estimation of unfiltered faces. IEEE Trans Inf Forensics Secur 9:2170–2179

    Google Scholar 

  26. Ahmed HA, Rashid TA, Sidiq A (2016) Face behavior recognition through support vector machines. Int J Adv Comput Sci Appl 7:101–108

    Google Scholar 

  27. Sun N, Zheng W, Sun C, Zou C, Zhao L (2006) Gender classification based on boosting local binary pattern. In: International symposium on neural networks, pp 194–201

  28. Shan C (2012) Learning local binary patterns for gender classification on real-world face images. Pattern Recogn Lett 33:431–437

    Google Scholar 

  29. Wang J-G, Li J, Yau W-Y, Sung E (2010) Boosting dense SIFT descriptors and shape contexts of face images for gender recognition. In: 2010 IEEE computer society conference on computer vision and pattern recognition-workshops, pp 96–102

  30. Bekios-Calfa J, Buenaposada JM, Baumela L (2014) Robust gender recognition by exploiting facial attributes dependencies. Pattern Recogn Lett 36:228–234

    Google Scholar 

  31. Alexandre LA (2010) Gender recognition: a multiscale decision fusion approach. Pattern Recogn Lett 31:1422–1427

    Google Scholar 

  32. Patel B, Maheshwari R, Raman B (2016) Compass local binary patterns for gender recognition of facial photographs and sketches. Neurocomputing 218:203–215

    Google Scholar 

  33. Li X, Zhao X, Fu Y, Liu Y (2010) Bimodal gender recognition from face and fingerprint. In: 2010 IEEE computer society conference on computer vision and pattern recognition, pp 2590–2597

  34. Bekhouche SE, Ouafi A, Dornaika F, Taleb-Ahmed A, Hadid A (2017) Pyramid multi-level features for facial demographic estimation. Expert Syst Appl 80:297–310

    Google Scholar 

  35. Divate CP, Ali SZ (2018) Study of different bio-metric based gender classification systems. In: International conference on inventive research in computing applications (ICIRCA), pp 347–353

  36. Ali AM, Rashid TA (2016) Kernel visual keyword description for object and place recognition. In: Advances in signal processing and intelligent recognition systems. Springer, Cham, pp 27–38

    Google Scholar 

  37. Moghaddam B, Yang M-H (2002) Learning gender with support faces. IEEE Trans Pattern Anal Mach Intell 24:707–711

    Google Scholar 

  38. Bekios-Calfa J, Buenaposada JM, Baumela L (2010) Revisiting linear discriminant techniques in gender recognition. IEEE Trans Pattern Anal Mach Intell 33:858–864

    Google Scholar 

  39. Duan M, Li K, Yang C, Li K (2018) A hybrid deep learning CNN–ELM for age and gender classification. Neurocomputing 275:448–461

    Google Scholar 

  40. Dhomne A, Kumar R, Bhan V (2018) Gender recognition through face using deep learning. Procedia Comput Sci 132:2–10

    Google Scholar 

  41. Zhang K, Tan L, Li Z, Qiao Y (2016) Gender and smile classification using deep convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 34–38

  42. Mansanet J, Albiol A, Paredes R (2016) Local deep neural networks for gender recognition. Pattern Recogn Lett 70:80–86

    Google Scholar 

  43. Asmara RA, Masruri I, Rahmad C, Siradjuddin I, Rohadi E, Ronilaya F et al (2018) Comparative study of gait gender identification using gait energy image (GEI) and gait information image (GII). In: MATEC web of conferences, p 15006

  44. Hassan OMS, Abdulazeez AM, Tiryaki VM (2018) Gait-Based human gender classification using lifting 5/3 wavelet and principal component analysis. In: International conference on advanced science and engineering (ICOASE), pp 173–178

  45. Bei S, Deng J, Zhen Z, Shaojing S (2019) Gender recognition via fused silhouette features based on visual sensors. IEEE Sens J 19(20):9496–9503

    Google Scholar 

  46. Amayeh G, Bebis G, Nicolescu M (2008) Gender classification from hand shape. In: IEEE computer society conference on computer vision and pattern recognition workshops, pp 1–7

  47. Matsumoto R, Yoshimura H, Nishiyama M, Iwai Y (2017) Feature extraction using gaze of participants for classifying gender of pedestrians in images. In: IEEE international conference on image processing (ICIP), pp 3545–3549

  48. Liu T, Ye X, Sun B (2018) Clothing and carrying invariant gait-based gender recognition. In: International conference on image and video processing, and artificial intelligence, p 108360X

  49. Cao L, Dikmen M, Fu Y, Huang TS (2008) Gender recognition from body. In: Proceedings of the 16th ACM international conference on multimedia, pp 725–728

  50. Collins M, Zhang J, Miller P, Wang H (2009) Full body image feature representations for gender profiling. In: IEEE 12th international conference on computer vision workshops, ICCV workshops, pp 1235–1242

  51. Geelen CD, Wijnhoven RG, Dubbelman G (2015) Gender classification in low-resolution surveillance video: in-depth comparison of random forests and SVMs. In: Video surveillance and transportation imaging applications, p 94070 M

  52. Ng C-B, Tay Y-H, Goi B-M (2013) Comparing image representations for training a convolutional neural network to classify gender. In: 1st international conference on artificial intelligence, modelling and simulation, pp 29–33

  53. Antipov G, Berrani S-A, Ruchaud N, Dugelay J-L (2015) Learned vs. hand-crafted features for pedestrian gender recognition. In: Proceedings of the 23rd ACM international conference on multimedia, pp 1263–1266

  54. Raza M, Zonghai C, Rehman SU, Zhenhua G, Jikai W, Peng B (2017) Part-wise pedestrian gender recognition via deep convolutional neural networks, pp 26–6

  55. Ng C-B, Tay Y-H, Goi B-M (2017) Training strategy for convolutional neural networks in pedestrian gender classification. In: Second international workshop on pattern recognition, p 104431A

  56. Raza M, Sharif M, Yasmin M, Khan MA, Saba T, Fernandes SL (2018) Appearance based pedestrians’ gender recognition by employing stacked auto encoders in deep learning. Future Gen Comput Syst 88:28–39

    Google Scholar 

  57. Cai L, Zhu J, Zeng H, Chen J, Cai C (2018) Deep-learned and hand-crafted features fusion network for pedestrian gender recognition. In: Proceedings of ELM-2016. Springer, Berlin, pp 207–215

  58. Cai L, Zhu J, Zeng H, Chen J, Cai C, Ma K-K (2018) Hog-assisted deep feature learning for pedestrian gender recognition. J Franklin Inst 355:1991–2008

    Google Scholar 

  59. Ng CB, Tay Y-H, Goi B-M (2018) Pedestrian gender classification using combined global and local parts-based convolutional neural networks. Pattern Anal Appl 22(4):1469–1480

    MathSciNet  Google Scholar 

  60. Sindagi VA, Patel VM (2018) A survey of recent advances in cnn-based single image crowd counting and density estimation. Pattern Recogn Lett 107:3–16

    Google Scholar 

  61. Li C, Guo J, Porikli F, Pang Y (2018) Lightennet: a convolutional neural network for weakly illuminated image enhancement. Pattern Recogn Lett 104:15–22

    Google Scholar 

  62. Rashid M, Khan MA, Sharif M, Raza M, Sarfraz MM, Afza F (2019) Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features. Multimedia Tools Appl 78:15751–15777

    Google Scholar 

  63. Khan MA, Akram T, Sharif M, Awais M, Javed K, Ali H et al (2018) CCDF: automatic system for segmentation and recognition of fruit crops diseases based on correlation coefficient and deep CNN features. Comput Electron Agric 155:220–236

    Google Scholar 

  64. Sharif M, Attique Khan M, Rashid M, Yasmin M, Afza F, Tanik UJ (2019) Deep CNN and geometric features-based gastrointestinal tract diseases detection and classification from wireless capsule endoscopy images. J Exp Theor Artif Intell, 1–23

  65. Rashid TA, Fattah P, Awla DK (2018) Using accuracy measure for improving the training of lstm with metaheuristic algorithms. Procedia Computer Science 140:324–333

    Google Scholar 

  66. Rashid TA (2016) Convolutional neural networks based method for improving facial expression recognition. In: The international symposium on intelligent systems technologies and applications, pp 73–84

  67. Shamsaldin AS, Fattah P, Rashid TA, Al-Salihi NK (2019) A study of the convolutional neural networks applications. UKH J Sci Eng 3:31–40

    Google Scholar 

  68. Rashid TA, Abbas DK, Turel YK (2019) A multi hidden recurrent neural network with a modified grey wolf optimizer. PLoS ONE 14:e0213237

    Google Scholar 

  69. Rashid TA, Abdullah SM (2018) A hybrid of artificial bee colony, genetic algorithm, and neural network for diabetic mellitus diagnosing. ARO Sci J Koya Univ 6:55–64

    Google Scholar 

  70. Uddin MA, Lee Y-K (2019) Feature fusion of deep spatial features and handcrafted spatiotemporal features for human action recognition. Sensors 19:1599

    Google Scholar 

  71. Fan X, Tjahjadi T (2019) Fusing dynamic deep learned features and handcrafted features for facial expression recognition. J Vis Commun Image Represent 65:102659

    Google Scholar 

  72. Georgescu M-I, Ionescu RT, Popescu M (2019) Local learning with deep and handcrafted features for facial expression recognition. IEEE Access 7:64827–64836

    Google Scholar 

  73. Hasan AM, Jalab HA, Meziane F, Kahtan H, Al-Ahmad AS (2019) Combining deep and handcrafted image features for MRI brain scan classification. IEEE Access 7:79959–79967

    Google Scholar 

  74. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), vol 1. IEEE, pp 886–893

  75. Wang X, Zhao C, Miao D, Wei Z, Zhang R, Ye T (2016) Fusion of multiple channel features for person re-identification. Neurocomputing 213:125–136

    Google Scholar 

  76. Qi Z, Tian Y, Shi Y (2013) Efficient railway tracks detection and turnouts recognition method using HOG features. Neural Comput Appl 23:245–254

    Google Scholar 

  77. Chee KW, Teoh SS (2019) Pedestrian detection in visual images using combination of HOG and HOM features. In: 10th international conference on robotics, vision, signal processing and power applications, pp. 591–597

  78. Wei Y, Tian Q, Guo J, Huang W, Cao J (2019) Multi-vehicle detection algorithm through combining Harr and HOG features. Math Comput Simul 155:130–145

    MathSciNet  Google Scholar 

  79. Firuzi K, Vakilian M, Phung BT, Blackburn TR (2018) Partial discharges pattern recognition of transformer defect model by LBP & HOG features. IEEE Trans Power Delivery 34:542–550

    Google Scholar 

  80. Xiao T, Li S, Wang B, Lin L, Wang X (2017) Joint detection and identification feature learning for person search. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3415–3424

  81. Xu J, Luo L, Deng C, Huang H (2018) Bilevel distance metric learning for robust image recognition. In: Advances in neural information processing systems, pp 4198–4207

    Google Scholar 

  82. Junejo IN (2019) A deep learning based multi-color space approach for pedestrian attribute recognition. In: Proceedings of the 2019 3rd international conference on graphics and signal processing, pp 113–116

  83. Ren Q-Q, Tian W-D, Zhao Z-Q (2019) “Person re-identification based on feature fusion. In: International Conference on Intelligent Computing, 2019, pp. 65-73

  84. Liao S, Zhao G, Kellokumpu V, Pietikäinen M, Li SZ (2010) Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: IEEE computer society conference on computer vision and pattern recognition, pp 1301–1306

  85. AroraS, Bhatia M (2018) A robust approach for gender recognition using deep learning. In: 9th international conference on computing, communication and networking technologies (ICCCNT), pp 1–6

  86. Labati RD, Muñoz E, Piuri V, Sassi R, Scotti F (2018) Deep-ECG: convolutional neural networks for ECG biometric recognition. Pattern Recognit Lett 126:78–85

    Google Scholar 

  87. Cheng E-J, Chou K-P, Rajora S, Jin B-H, Tanveer M, Lin C-T et al (2019) Deep Sparse Representation Classifier for facial recognition and detection system. Pattern Recognit Lett 125:71–77

    Google Scholar 

  88. Fayyaz M, Yasmin M, Sharif M, Shah JH, Raza M, Iqbal T (2019) Person re-identification with features-based clustering and deep features. Neural Comput Appl, 1–22

  89. Hu F, Xia G-S, Hu J, Zhang L (2015) Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery. Remote Sens 7:14680–14707

    Google Scholar 

  90. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  91. Nigam K, Lafferty J, McCallum A (1999) Using maximum entropy for text classification. In: IJCAI-99 workshop on machine learning for information filtering, pp 61–67

  92. Morais CL, Lima KM, Martin FL (2019) Uncertainty estimation and misclassification probability for classification models based on discriminant analysis and support vector machines. Anal Chim Acta 1063:40–46

    Google Scholar 

  93. Radhika K, Varadarajan S (2018) Ensemble subspace discriminant classification of satellite images

  94. Shirkhorshidi AS, Aghabozorgi S, Wah TY (2015) A comparison study on similarity and dissimilarity measures in clustering continuous data. PLoS One 10(12)

  95. Lekdioui K, Messoussi R, Ruichek Y, Chaabi Y, Touahni R (2017) Facial decomposition for expression recognition using texture/shape descriptors and SVM classifier. Sig Process Image Commun 58:300–312

    Google Scholar 

  96. Niu X-X, Suen CY (2012) A novel hybrid CNN–SVM classifier for recognizing handwritten digits. Pattern Recognit 45:1318–1325

    Google Scholar 

  97. Deng Y, Luo P, Loy CC, Tang X (2014) Pedestrian attribute recognition at far distance. In: Proceedings of the 22nd ACM international conference on multimedia, pp 789–792

  98. Zhu W, Miao J, Qing L, Huang G-B (2015) Hierarchical extreme learning machine for unsupervised representation learning. In: International joint conference on neural networks (IJCNN), pp 1–8

  99. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  100. Wang Q, Ye L, Luo H, Men A, Zhao F, Huang Y (2019) Pedestrian stride-length estimation based on LSTM and denoising autoencoders. Sensors 19:840

    Google Scholar 

  101. Rashid T, Jabar A (2018) A modified particle swarm optimization with neural network via Euclidean distance. Int J Recent Contrib Eng Sci IT (IJES) 6:4–18

    Google Scholar 

  102. Haghighat M, Abdel-Mottaleb M, Alhalabi W (2016) Discriminant correlation analysis: real-time feature level fusion for multimodal biometric recognition. IEEE Trans Inf Forensics Secur 11:1984–1996

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, COMSATS University Islamabad, Wah Campus, Wah Cantt, Pakistan

    Muhammad Fayyaz, Mussarat Yasmin, Muhammad Sharif & Mudassar Raza

Authors
  1. Muhammad Fayyaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mussarat Yasmin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mudassar Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Fayyaz.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fayyaz, M., Yasmin, M., Sharif, M. et al. J-LDFR: joint low-level and deep neural network feature representations for pedestrian gender classification. Neural Comput & Applic 33, 361–391 (2021). https://doi.org/10.1007/s00521-020-05015-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05015-1

Keywords

Search

Navigation