Skip to main content
SpringerLink
Log in

Efficient deep feature selection for remote sensing image recognition with fused deep learning architectures

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Convolutional neural networks (CNNs) have recently emerged as a popular topic for machine learning in various academic and industrial fields. It is often an important problem to obtain a dataset with an appropriate size for CNN training. However, the lack of training data in the case of remote image research leads to poor performance due to the overfitting problem. In addition, the back-propagation algorithm used in CNN training is usually very slow and thus requires tuning different hyper-parameters. In order to overcome these drawbacks, a new approach fully based on machine learning algorithm to learn useful CNN features from Alexnet, VGG16, VGG19, GoogleNet, ResNet and SqueezeNet CNN architectures is proposed in the present study. This method performs a fast and accurate classification suitable for recognition systems. Alexnet, VGG16, VGG19, GoogleNet, ResNet and SqueezeNet pretrained architectures were used as feature extractors. The proposed method obtains features from the last fully connected layers of each architecture and applies the ReliefF feature selection algorithm to obtain efficient features. Then, selected features are given to the support vector machine classifier with the CNN-learned features instead of the FC layers of CNN to obtain excellent results. The effectiveness of the proposed method was tested on the UC-Merced dataset. Experimental results demonstrate that the proposed classification method achieved an accuracy rate of 98.76% and 99.29% in 50% and 80% training experiment, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Kobayashi T (2014) Dirichlet-based histogram feature transform for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3278–3285

  2. Negrel R, Picard D, Gosselin PH (2014) Evaluation of second-order visual features for land-use classification. In: 2014 12th International Workshop on Content-Based Multimedia Indexing (CBMI), pp 1–5. IEEE

  3. Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp 270–279. ACM

  4. Hu F, Xia GS, Wang Z, Huang X, Zhang L, Sun H (2015) Unsupervised feature learning via spectral clustering of multidimensional patches for remotely sensed scene classification. IEEE J Sel Top Appl Earth Obs Remote Sens 8:5

    Google Scholar 

  5. Chen C, Zhang B, Su H, Li W, Wang L (2016) Land-use scene classification using multi-scale completed local binary patterns. SIViP 10(4):745–752

    Article  Google Scholar 

  6. Chen S, Tian Y (2014) Pyramid of spatial relatons for scene-level land use classification. IEEE Trans Geosci Remote Sens 53(4):1947–1957

    Article  MathSciNet  Google Scholar 

  7. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36(4):193–202

    Article  Google Scholar 

  8. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp 675–678. ACM

  9. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105

  10. Qassim H, Verma A, Feinzimer D (2018) Compressed residual-VGG16 CNN model for big data places image recognition. In: 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), pp 169–175. IEEE

  11. Mateen M, Wen J, Song S, Huang Z (2019) Fundus image classification using VGG-19 architecture with PCA and SVD. Symmetry 11(1):1

    Article  Google Scholar 

  12. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  14. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50 × fewer parameters and < 0.5 MB model size. arXiv preprint arXiv:1602.07360

  15. Kutlu H, Avcı E (2019) A novel method for classifying liver and brain tumors using convolutional neural networks, discrete wavelet transform and long short-term memory networks. Sensors 19(9):1992

    Article  Google Scholar 

  16. Kutlu H, Avci E, Özyurt F (2019) White blood cells detection and classification based on regional convolutional neural networks. Med Hypotheses 135:109472

    Article  Google Scholar 

  17. Penatti OA, Nogueira K, Dos Santos JA (2015) Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 44–51

  18. Özyurt F, Sert E, Avci E, Dogantekin E (2019) Brain tumor detection based on convolutional neural network with neutrosophic expert maximum fuzzy sure entropy. Measurement 147:106830

    Article  Google Scholar 

  19. Özyurt F, Sert E, Avci D (2019) An expert system for brain tumor detection: fuzzy C-means with super resolution and convolutional neural network with extreme learning machine. Med Hypotheses 134:109433

    Article  Google Scholar 

  20. Özyurt F (2020) A fused CNN model for WBC detection with MRMR feature selection and extreme learning machine. Soft Comput 83:1–10

    Google Scholar 

  21. Understanding of Convolutional Neural Network (CNN) — Deep Learning. https://medium.com/@RaghavPrabhu/understanding-of-convolutional-neural-network-cnn-deep-learning-99760835f148. Accessed 19 May 2019

  22. Sert E, Özyurt F, Doğantekin A (2019) A new approach for brain tumor diagnosis system: single image super resolution based maximum fuzzy entropy segmentation and convolutional neural network. Med Hypotheses 133:109413

    Article  Google Scholar 

  23. Lv X, Ming D, Chen Y, Wang M (2019) Very high resolution remote sensing image classification with SEEDS-CNN and scale effect analysis for superpixel CNN classification. Int J Remote Sens 40(2):506–531

    Article  Google Scholar 

  24. Bouvrie J (2006) Notes on Convolutional Neural Networks [Online]. Available: http://cogprints.org/5869/1/cnn_tutorial.pdf

  25. Kira K, Rendell LA (1992) The feature selection problem: traditional methods and a new algorithm. Aaaı 2:129–134

    Google Scholar 

  26. Kononenko I (1994) Estimating attributes: analysis and extensions of RELIEF. In: Bergadano F, De Raedt L (eds) Machine Learning: ECML-94. ECML 1994. Lecture Notes on Computer Science (Lecture Notes in Artificial Intelligence), vol 784. Springer, Berlin, Heidelberg

  27. Tuncer T, Ertam F (2019) Neighborhood component analysis and reliefF based survival recognition methods for Hepatocellular carcinoma. Physica A Stat Mech Appl 12:123143

    Google Scholar 

  28. Soman KP, Loganathan R, Ajay V (2009) Machine learning with SVM and other kernel methods. PHI Learning Pvt. Ltd., New Delhi

    Google Scholar 

  29. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  30. Vapnik V, Levin E, Le Cun Y (1994) Measuring the VC-dimension of a learning machine. Neural Comput 6(5):851–876

    Article  Google Scholar 

  31. Kindermann J, Paaß G, Leopold E (2001) Error correcting codes with optimized Kullback-Leibler distances for text categorization. In: European Conference on Principles of Data Mining and Knowledge Discovery. Springer, Berlin, Heidelberg, pp 266–276

  32. Pierna JF, Baeten V, Renier AM, Cogdill RP, Dardenne P (2004) Combination of support vector machines (SVM) and near-infrared (NIR) imaging spectroscopy for the detection of meat and bone meal (MBM) in compound feeds. J Chemom A J Chemom Soc 18(7–8):341–349

    Google Scholar 

  33. Yang Z, Wang C, Zhang Z, Li J (2019) Mini-batch algorithms with online step size. Knowl Based Syst 165:228–240

    Article  Google Scholar 

  34. Ren J, Jiang X, Yuan J (2015) Learning LBP structure by maximizing the conditional mutual information. Pattern Recogn 48(10):3180–3190

    Article  Google Scholar 

  35. Cheriyadat AM (2014) Unsupervised feature learning for aerial scene classification. IEEE Trans Geosci Remote Sens 52:439–451

    Article  Google Scholar 

  36. Zhang F, Du B, Zhang L (2016) Scene classification via a gradient boosting random convolutional network framework. IEEE Trans Geosci Remote Sens 54:1793–1802

    Article  Google Scholar 

  37. Zou J, Li W, Chen C, Du Q (2016) Scene classification using local and global features with collaborative representation fusion. Inf Sci 348:209–226

    Article  MathSciNet  Google Scholar 

  38. Liu Y, Zhong Y, Fei F, Zhu Q, Qin Q (2018) Scene classification based on a deep random-scale stretched convolutional neural network. Remote Sens 10:444

    Article  Google Scholar 

  39. Shao W, Yang W, Xia GS, Liu G (2013) A hierarchical scheme of multiple feature fusion for high-resolution satellite scene categorization. In: International Conference on Computer Vision Systems. Springer, Berlin, Heidelberg, pp 324–333

  40. Xia GS, Hu J, Hu F, Shi B, Bai X, Zhong Y, Lu X (2017) AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Trans Geosci Remote Sens 55(7):3965–3981

    Article  Google Scholar 

  41. Castelluccio M, Poggi G, Sansone C, Verdoliva L (2015) Land use classification in remote sensing images by convolutional neural networks. arXiv preprint arXiv:1508.00092

  42. Hu F, Xia GS, Hu J, Zhang L (2015) Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery. Remote Sens 7(11):14680–14707

    Article  Google Scholar 

  43. Bian X, Chen C, Tian L, Du Q (2017) Fusing local and global features for high-resolution scene classification. IEEE J Sel Top Appl Earth Obs Remote Sens 10(6):2889–2901

    Article  Google Scholar 

  44. Anwer RM, Khan FS, van de Weijer J, Molinier M, Laaksonen J (2018) Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. ISPRS J Photogramm Remote Sens 138:74–85

    Article  Google Scholar 

  45. Chaib S, Liu H, Gu Y, Yao H (2017) Deep feature fusion for VHR remote sensing scene classification. IEEE Trans Geosci Remote Sens 55(8):4775–4784

    Article  Google Scholar 

  46. Yu Y, Liu F (2018) A two-stream deep fusion framework for high-resolution aerial scene classification. Comput Intell Neurosci. https://doi.org/10.1155/2018/8639367

    Article  Google Scholar 

  47. Zeng D, Chen S, Chen B, Li S (2018) Improving remote sensing scene classification by integrating global-context and local-object features. Remote Sens 10(5):734

    Article  Google Scholar 

Download references

Funding

There is no funding source for this article.

Author information

Authors and Affiliations

  1. Department of Informatics, Firat University, Elazig, Turkey

    Fatih Özyurt

Authors
  1. Fatih Özyurt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fatih Özyurt.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest related to this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Özyurt, F. Efficient deep feature selection for remote sensing image recognition with fused deep learning architectures. J Supercomput 76, 8413–8431 (2020). https://doi.org/10.1007/s11227-019-03106-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-019-03106-y

Keywords

Search

Navigation