Skip to main content
SpringerLink
Log in

Residual-capsule networks with threshold convolution for segmentation of wheat plantation rows in UAV images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The early growth process of wheat is vulnerable to various factors, and poor growth leads to vacancies in the planting row. Therefore, the wheat images captured by unmanned aerial vehicles (UAV) are essential for monitoring the growth of wheat and preventing diseases and insect pests. This paper uses wheat images captured by UAV as a dataset, and propose a novel residual-capsule network with threshold convolution (RCTC) for segmentation of wheat plantation rows. The network is achieved by replacing the AveragePooling of the improved ResNet34 with Capsule. Since the capsule network represents the features by vectors, it can explain the direction of features and the relative positions between features. Therefore, deeper feature information can be extracted. In addition, to reduce redundant features and enhance effective features, a new threshold convolution is also proposed. Experiments on the wheat field dataset show that our proposed algorithm can effectively segment the wheat plantation rows images collected by UAV, and is superior to some existing well-known algorithms, and can provide scientific support and reference for the decision-making process of smart agriculture.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Cai W, Wei Z Remote Sensing Image Classification Based on a Cross-Attention Mechanism and Graph Convolution. IEEE Geosci Remote Sens Lett. https://doi.org/10.1109/LGRS.2020.3026587

  2. César Pereira Júnior P, Monteiro A, Da Luz Ribeiro R, Sobieranski AC, Von Wangenheim A (2020) Comparison of supervised classifiers and image features for crop rows segmentation on aerial images. Appl Artif Intell 34(4):271–291

    Article  Google Scholar 

  3. De Castro AI, Torres-Sánchez J, Peña JM, Jiménez-Brenes FM, Csillik O, López-Granados F (2018) An automatic random forest-OBIA algorithm for early weed mapping between and within crop rows using UAV imagery. Remote Sens 10(2):285

    Article  Google Scholar 

  4. Deng F, Pu S, Chen X, Shi Y, Yuan T, Pu S (2018) Hyperspectral image classification with capsule network using limited training samples. Sensors 18(9):3153

    Article  Google Scholar 

  5. Fareed N, Rehman K (2020) Integration of remote sensing and GIS to extract plantation rows from a drone-based image point cloud digital surface model. ISPRS Int J Geo Inf 9(3):151

    Article  Google Scholar 

  6. Feng A, Zhou J, Vories E, Sudduth KA (2020) Evaluation of cotton emergence using UAV-based imagery and deep learning. Comput Electron Agric 177:105711

    Article  Google Scholar 

  7. Ganchenko V, Doudkin A (2019) Image Semantic Segmentation Based on Convolutional Neural Networks for Monitoring Agricultural Vegetation. In: Ablameyko S, Krasnoproshin V, Lukashevich M (eds) Pattern Recognition and Information Processing. PRIP 2019. Communications in Computer and Information Science, vol 1055. Springer, Cham. https://doi.org/10.1007/978-3-030-35430-5_5

  8. Gao H, Yang Y, Yao D, Li C (2019) Hyperspectral image classification with pre-activation residual attention network. IEEE Access 7:176587–176599

    Article  Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778)

  10. Hu WJ, Fan J, Du YX, Li BS, Xiong N, Bekkering E (2020) MDFC–ResNet: an agricultural IoT system to accurately recognize crop diseases. IEEE Access 8:115287–115298

    Article  Google Scholar 

  11. Huang Z, Zhang Y, Li Q, Zhang T, Sang N, Hong H (May 2018) Progressive dual-domain filter for enhancing and Denoising optical remote-sensing images. IEEE Geosci Remote Sens Lett 15(5):759–763

    Article  Google Scholar 

  12. Huang Z, Zhang Y, Li Q, Li X, Zhang T, Sang N, Hong H (Oct. 2020) Joint analysis and weighted synthesis sparsity priors for simultaneous Denoising and Destriping optical remote sensing images. IEEE Trans Geosci Remote Sens 58(10):6958–6982

    Article  Google Scholar 

  13. Jeon HY, Tian LF, Zhu H Robust crop and weed segmentation under uncontrolled outdoor illumination. Sensors 11(6):6270–6283

  14. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  15. Kurup RV, Anupama MA, Vinayakumar R, Sowmya V, Soman KP (2019) Capsule network for plant disease and plant species classification. In international conference on computational vision and bio inspired computing (pp. 413-421)

  16. Li Y, Qian M, Liu P, Cai Q, Li X, Guo J, … Qin L (2019) The recognition of rice images by UAV based on capsule network. Clust Comput 22(4):9515–9524

    Article  Google Scholar 

  17. Li K et al (2020) Attention-Guided Multi-Scale Segmentation Neural Network for Interactive Extraction of Region Objects from High-Resolution Satellite Imagery. Remote Sens 12.5:789

    Article  Google Scholar 

  18. Lin Aung H et al. (2020) Farm Parcel Delineation Using Spatio-Temporal Convolutional Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops

  19. Lottes P, Behley J, Milioto A, Stachniss C (2018) Fully convolutional networks with sequential information for robust crop and weed detection in precision farming. IEEE Robotics Automation Lett 3(4):2870–2877

    Article  Google Scholar 

  20. Osco LP, de Arruda MDS, Junior JM, da Silva NB, Ramos APM, Moryia ÉAS, … Gonçalves WN (2020) A convolutional neural network approach for counting and geolocating citrus-trees in UAV multispectral imagery. ISPRS J Photogramm Remote Sens 160:97–106

    Article  Google Scholar 

  21. Paoletti ME, Haut JM, Fernandez-Beltran R, Plaza J, Plaza AJ, Pla F (2018) Deep pyramidal residual networks for spectral–spatial hyperspectral image classification. IEEE Trans Geosci Remote Sens 57(2):740–754

    Article  Google Scholar 

  22. Rocha BM, da Silva Vieira G, Fonseca AU, Pedrini H, de Sousa NM, Soares F (2020) Evaluation and detection of gaps in curved sugarcane planting lines in aerial images. In 2020 IEEE Canadian conference on electrical and computer engineering (CCECE), pp 1–4. https://doi.org/10.1109/CCECE47787.2020.9255701

  23. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In advances in neural information processing systems (pp. 3856-3866)

  24. Salman M, Yüksel SE (2018) Fusion of hyperspectral image and LiDAR data and classification using deep convolutional neural networks. In 2018 26th signal processing and communications applications conference (SIU).; pp. 1-4.

  25. Tang YC, Wang C, Luo L, Zou X (2020) Recognition and localization methods for vision-based fruit picking robots: a review. Front Plant Sci 11:510

    Article  Google Scholar 

  26. Wang A, Xu Y, Wei X, Cui B (2020) Semantic segmentation of crop and weed using an encoder-decoder network and image enhancement method under uncontrolled outdoor illumination. IEEE Access 8:81724–81734

    Article  Google Scholar 

  27. Wang Z, Zou C, Cai W (2020) Small sample classification of hyperspectral remote sensing images based on sequential joint Deeping learning model. IEEE Access 8:71353–71363

    Article  Google Scholar 

  28. Xu Q, Chen K, Sun X, Zhang Y, Li H, Xu G (2020) Pseudo-Siamese capsule network for aerial remote sensing images change detection. IEEE Geosci Remote Sens Lett:1–5

  29. Yang G, Gewali UB, Ientilucci E, Gartley M, Monteiro ST (2018) Dual-channel DenseNet for hyperspectral image classification. In IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium.; pp. 2595–2598

  30. Yang Q, Liu M, Zhang Z, Yang S, Ning J, Han W (2019) Mapping plastic mulched farmland for high resolution images of unmanned aerial vehicle using deep semantic segmentation. Remote Sens 11(17):2008

    Article  Google Scholar 

  31. You H, Tian S, Yu L, Lv Y (2019) Pixel-level remote sensing image recognition based on bidirectional word vectors. IEEE Trans Geosci Remote Sens 58(2):1281–1293

    Article  Google Scholar 

  32. Zhang C, Wan S, Gao S, Yu F, Wei Q, Wang G, ..., Song D (2017) A Segmentation Model for Extracting Farmland and Woodland from Remote Sensing Image

  33. Zheng H, Zhou X, He J, Yao X, Cheng T, Zhu Y, Cao W, Tian Y (2020) Early season detection of rice plants using RGB, NIR-GB and multispectral images from unmanned aerial vehicle (UAV). Comput Electron Agric 169:105223

    Article  Google Scholar 

Download references

Acknowledgments

This research was funded by the Hunan Key Laboratory of Intelligent Logistics Technology under Grant 2019TP1015.

Author information

Authors and Affiliations

  1. Central South University of Forestry and Technology, Changsha, 410004, China

    Weiwei Cai, Zhanguo Wei, Yaping Song & Meilin Li

  2. Changsha Astra Information Technology Co., Ltd., Changsha, 410219, China

    Weiwei Cai

  3. Hefei University, Hefei, 230009, China

    Xuechun Yang

Authors
  1. Weiwei Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhanguo Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaping Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Meilin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuechun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhanguo Wei.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, W., Wei, Z., Song, Y. et al. Residual-capsule networks with threshold convolution for segmentation of wheat plantation rows in UAV images. Multimed Tools Appl 80, 32131–32147 (2021). https://doi.org/10.1007/s11042-021-11203-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11203-5

Keywords

Search

Navigation