Skip to main content
SpringerLink
Log in

A mixed depthwise separation residual network for image feature extraction

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Aiming for the ResNet network structure used for feature extraction, there are still problems of overfitting, slow calculation speed and room for improvement of accuracy. A mixed depthwise separation residual network is proposed for image feature extraction. The residual and inverted residual blocks are used in the research, which makes the three kinds of residual units iteratively interact in the training process. This process promotes the feature representation of deep learning neural networks and alleviates the problem of feature dispersion caused by the deep network level in traditional expression recognition research. The experiment used CK+ and fer2013 datasets. The synthesis of multiple experimental results shows that the optimized deep learning model has an accuracy rate of about 2.8% higher than that of ResNet-50 on the Fer2013 test set. Compared with traditional residual network structure, it verifies that the network proposed in this paper has better effects in the field of facial expression recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ferro, E., Gennaro, C., Nordio, A., et al. (2020). 5G-enabled security scenarios for unmanned aircraft: Experimentation in urban environment. Drones, 4(2), 22.

    Article  Google Scholar 

  2. Qiu, J., Du, L., Chen, Y., et al. (2020). Artificial intelligence security in 5G networks: Adversarial examples for estimating a travel time task. IEEE Vehicular Technology Magazine, 15(3), 95–100.

    Article  Google Scholar 

  3. Li, D., Wang, Z., Gao, Q., et al. (2019). Facial expression recognition based on electroencephalogram and facial landmark localization. Technology and Health Care, 27(4), 373–387.

    Article  Google Scholar 

  4. Zhang, T. (2017). Facial expression recognition based on deep learning: A survey. In International conference on intelligent and interactive systems and applications (pp. 345–352). Cham: Springer.

  5. Zeng, J., Shan, S., & Chen, X. (2018). Facial expression recognition with inconsistently annotated datasets. In Proceedings of the European conference on computer vision (ECCV) (pp. 222–237).

  6. Shan, C., Gong, S., & McOwan, P. W. (2009). Facial expression recognition based on local binary patterns: A comprehensive study. Image and vision Computing, 27(6), 803–816.

    Article  Google Scholar 

  7. Yang, H., Ciftci, U., & Yin, L. (2018). Facial expression recognition by de-expression residue learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2168–2177).

  8. Gogić, I., Manhart, M., Pandžić, I. S., et al. (2020). Fast facial expression recognition using local binary features and shallow neural networks. The Visual Computer, 36(1), 97–112.

    Article  Google Scholar 

  9. Liu, P., Han, S., Meng, Z., et al. (2014). Facial expression recognition via a boosted deep belief network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1805–1812).

  10. Yu, Z., & Zhang, C. (2015). Image based static facial expression recognition with multiple deep network learning. In Proceedings of the 2015 ACM on international conference on multimodal interaction (pp. 435–442).

  11. Mollahosseini, A., Chan, D., & Mahoor, M. H. (2016). Going deeper in facial expression recognition using deep neural networks. In 2016 IEEE Winter conference on applications of computer vision (WACV) (pp. 1–10). IEEE.

  12. Orhan, A. E., & Pitkow, X. (2017). Skip connections eliminate singularities. arXiv preprint https://arxiv.org/abs/1701.09175.

  13. He, K., Zhang, X., Ren, S., et al. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).

  14. Sandler, M., Howard, A., Zhu, M., et al. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4510–4520).

  15. Chollet, F. (2017). Xception: Deep learning with depth wise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251–1258).

  16. Hua, B. S., Tran, M. K., & Yeung, S. K. (2018). Pointwise convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 984–993).

  17. Jiang, H., & Learned-Miller, E. (2017). Face detection with the faster R-CNN. In 2017 12th IEEE international conference on automatic face and gesture recognition (FG 2017) (pp. 650–657), IEEE.

  18. Zhang, J., Wu, X., Hoi, S. C. H., et al. (2020). Feature agglomeration networks for single stage face detection. Neurocomputing, 380, 180–189.

    Article  Google Scholar 

  19. Wang, S., Pan, B., Chen, H., et al. (2018). Thermal augmented expression recognition. IEEE Transactions on Cybernetics, 48(7), 2203–2214.

    Article  Google Scholar 

  20. Chen, A., Xing, H., & Wang, F. (2020). A facial expression recognition method using deep convolutional neural networks based on edge computing. IEEE Access, 8, 49741–49751.

    Article  Google Scholar 

  21. Jiao, Y., Jia, X., & Zhao, J. (2019). Facial expression recognition method based on difference center-symmetric local directional pattern. In 2019 International conference on computer, network, communication and information systems (CNCI 2019). Atlantis Press.

  22. Nie, Z. (2020). Research on facial expression recognition of robot based on CNN convolution neural network. In 2020 IEEE international conference on power, intelligent computing and systems (ICPICS) (pp. 1067–1070). IEEE.

  23. Qin, S., Zhu, Z., Zou, Y., et al. (2020). Facial expression recognition based on Gabor wavelet transform and 2-channel CNN. International Journal of Wavelets, Multiresolution and Information Processing, 18(02), 2050003.

    Article  Google Scholar 

  24. Muthukrishnan, A., Kumar, D. V., & Kanagaraj, M. (2019). Internet of image things-discrete wavelet transform and Gabor wavelet transform based image enhancement resolution technique for IoT satellite applications. Cognitive Systems Research, 57, 46–53.

    Article  Google Scholar 

  25. Wang, C., Li, Z., Dey, N., et al. (2018). Histogram of oriented gradient based plantar pressure image feature extraction and classification employing fuzzy support vector machine. Journal of Medical Imaging and Health Informatics, 8(4), 842–854.

    Article  Google Scholar 

  26. Zhang, Y., Zhang, Z., Qin, J., et al. (2018). Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction. Pattern Recognition, 76, 662–678.

    Article  Google Scholar 

  27. Zhang, P., He, H., & Gao, L. (2019). A nonlinear and explicit framework of supervised manifold-feature extraction for hyperspectral image classification. Neurocomputing, 337, 315–324.

    Article  Google Scholar 

  28. Du, L., & Hu, H. (2019). Weighted patch-based manifold regularization dictionary pair learning model for facial expression recognition using iterative optimization classification strategy. Computer Vision and Image Understanding, 186, 13–24.

    Article  Google Scholar 

  29. Happy, S. L., Dantcheva, A., & Routray, A. (2019). Dual-threshold based local patch construction method for manifold approximation and its application to facial expression analysis. In 2019 27th European signal processing conference (EUSIPCO) (pp. 1–5). IEEE.

  30. Ghimire, D., Jeong, S., Lee, J., et al. (2017). Facial expression recognition based on local region specific features and support vector machines. Multimedia Tools and Applications, 76(6), 7803–7821.

    Article  Google Scholar 

  31. Tsai, H. H., & Chang, Y. C. (2018). Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Computing, 22(13), 4389–4405.

    Article  Google Scholar 

  32. Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2017). Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 10(1), 18–31.

    Article  Google Scholar 

  33. An, F., & Liu, Z. (2020). Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM. The Visual Computer, 36(3), 483–498.

    Article  Google Scholar 

  34. Ji, Y., Hu, Y., Yang, Y., et al. (2019). Cross-domain facial expression recognition via an intra-category common feature and inter-category distinction feature fusion network. Neurocomputing, 333, 231–239.

    Article  Google Scholar 

Download references

Acknowledgements

The study was supported by the Major Project of Natural Science Research of the Jiangsu Higher Education Institutions of China (18KJA520012), and the Xuzhou Science and Technology Plan Project (KC19197).

Author information

Authors and Affiliations

  1. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221000, Jiangsu, China

    Sanyou Zhang & Daihong Jiang

  2. Information and Electrical Engineering College, Xuzhou University of Technology, Xuzhou, 221000, Jiangsu, China

    Daihong Jiang & Cheng Yu

Authors
  1. Sanyou Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daihong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheng Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daihong Jiang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Jiang, D. & Yu, C. A mixed depthwise separation residual network for image feature extraction. Wireless Netw (2021). https://doi.org/10.1007/s11276-021-02665-4

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11276-021-02665-4

Keywords

Search

Navigation