Abstract
Vegetation indexes (VIs) are a key variable for monitoring the crop growth and estimating crop productivity. The spectral traits of the tassels were significantly different from those of the canopy leaves, which had an effect on the accuracy of using vegetation indices of canopy features to monitor crop growth. Advances in deep learning in computer vision provide opportunities for rapid and nondestructive tassels removal. The objective of this study was to remove tassels from images using a rapid and non-destructive method and to analyze and quantify the effect of tassels on canopy spectral information. This study proposed the color space transformation to divide tassels images into different growth stages and analyzed the performance of three convolutional neural networks (U-Net, PSPNet and DeeplabV3+) for tassel segmentation, and the analysis and quantification of the effect of tassels on canopy spectral information by calculating VIs and contributions. The results showed that the HSV (Hue, Saturation, Value) color space transformation divided images into three growth stages with robustness in the field environment. The U-Net model had the best segmentation precision (PA = 82.14%, MIoU = 84.43%). Moreover, the tassels significantly reduce canopy reflectance in the green wavelength range and the contribution of tassels to each of the VIs was different. In addition, the contribution of tassels to VIs is dynamic in different stages of tassels growth. This study provides a fast and non-destructive approach to quantify the effect of tassels on canopy spectral information in the field environment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bendig, J., Yu, K., Aasen, H., Bolten, A., Bennertz, S., Broscheit, J., et al. (2015). Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring in barley. International Journal of Applied Earth Observation and Geoinformation, 39, 79–87. https://doi.org/10.1016/j.jag.2015.02.012
Bolton, D. K., & Friedl, M. A. (2013). Forecasting crop yield using remotely sensed vegetation indices and crop phenology metrics. Agricultural and Forest Meteorology, 173, 74–84. https://doi.org/10.1016/j.agrformet.2013.01.007
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation (pp. 801–818). https://doi.org/10.1007/978-3-030-01234-2_49.
Chernov, V., Alander, J., & Bochko, V. (2015). Integer-based accurate conversion between RGB and HSV color spaces. Computers & Electrical Engineering, 46, 328–337. https://doi.org/10.1016/j.compeleceng.2015.08.005
Ciganda, V. S., Gitelson, A. A., & Schepers, J. (2012). How deep does a remote sensor sense? Expression of chlorophyll content in a maize canopy. Remote Sensing of Environment, 126, 240–247. https://doi.org/10.1016/j.rse.2012.08.019
Feng, W., Wu, Y., He, L., Ren, X., Wang, Y., Hou, G., et al. (2019). An optimized non-linear vegetation index for estimating leaf area index in winter wheat. Precision Agriculture, 20(6), 1157–1176. https://doi.org/10.1007/s11119-019-09648-8
Garcia, A. G., Escolano, S. O., Oprea, S., Martinez, V. V., & Guez, J. E. G. I. (2017). A review on deep learning techniques applied to semantic segmentation. CoRR. arXiv:1704.06857
Gitelson, A. A., Kaufman, Y. J., Stark, R., & Rundquist, D. (2002). Novel algorithms for remote estimation of vegetation fraction. Remote Sensing of Environment, 80(1), 76–87. https://doi.org/10.1016/S0034-4257(01)00289-9
Guijarro, M. I. A., Pajares, G., Riomoros, I., Herrera, P. J., Burgos-Artizzu, X. P., & Ribeiro, A. (2011). Automatic segmentation of relevant textures in agricultural images. Computers and Electronics in Agriculture, 75(1), 75–83. https://doi.org/10.1016/j.compag.2010.09.013
Hamuda, E., Mc Ginley, B., Glavin, M., & Jones, E. (2017). Automatic crop detection under field conditions using the HSV colour space and morphological operations. Computers and Electronics in Agriculture, 133, 97–107. https://doi.org/10.1016/j.compag.2016.11.021
He, Y., Zeng, H., Fan, Y., Ji, S., & Wu, J. (2019). Application of deep learning in integrated pest management: A real-time system for detection and diagnosis of oilseed rape pests. Mobile Information Systems, 2019, 1–14. https://doi.org/10.1155/2019/4570808
He, Y., Zhou, Z., Tian, L., Liu, Y., & Luo, X. (2020). Brown rice planthopper (Nilaparvata lugens Stal) detection based on deep learning. Precision Agriculture, 21(6), 1385–1402. https://doi.org/10.1007/s11119-020-09726-2
Herrmann, I., Bdolach, E., Montekyo, Y., Rachmilevitch, S., Townsend, P. A., & Karnieli, A. (2020). Assessment of maize yield and phenology by drone-mounted superspectral camera. Precision Agriculture, 21(1), 51–76. https://doi.org/10.1007/s11119-019-09659-5
Hunter, R. B., Daynard, T. B., Hume, D. J., Tanner, J. W., Curtis, J. D., & Kannenberg, L. W. (1969). Effect of tassel removal on grain yield of corn (Zea mays L.). Crop Science, 9(4), 405–406. https://doi.org/10.2135/cropsci1969.0011183X000900040003x
Jin, X., Zarco-Tejada, P., Schmidhalter, U., Reynolds, M. P., Hawkesford, M. J., Varshney, R. K., et al. (2020). High-throughput estimation of crop traits: A review of ground and aerial phenotyping platforms. IEEE Geoscience and Remote Sensing Magazine. https://doi.org/10.1109/MGRS.2020.2998816
Kurtulmu, F., & Kavdir, S. (2014). Detecting corn tassels using computer vision and support vector machines. Expert Systems with Applications, 41(16), 7390–7397. https://doi.org/10.1016/j.eswa.2014.06.013
Lambert, R. J., & Johnson, R. R. (1978). Leaf angle, tassel morphology, and the performance of maize hybrids 1. Crop Science, 18(3), 499–502. https://doi.org/10.2135/cropsci1978.0011183X001800030037x
Li, B., Junhua, B., Qing, X., Qinhuo, L., Ze, Z., Xin, L., & Linghua, M. (2019). Effects of maize tassel at the heading stage on radiation transfer characteristics of canopy reflectance. Transactions of the Chinese Society of Agricultural Engineering, 35(20), 162–170. https://doi.org/10.11975/j.issn.1002-6819.2019.20.020
Louhaichi, M., Borman, M. M., & Johnson, D. E. (2001). Spatially located platform and aerial photography for documentation of grazing impacts on wheat. Geocarto International, 16(1), 65–70. https://doi.org/10.1080/10106040108542184
Lu, H., Cao, Z., Xiao, Y., Fang, Z., & Zhu, Y. (2017). Towards fine-grained maize tassel flowering status recognition: Dataset, theory and practice. Applied Soft Computing, 56, 34–45. https://doi.org/10.1016/j.asoc.2017.02.026
Lu, H., Cao, Z., Xiao, Y., Li, Y., & Zhu, Y. (2015). Joint crop and tassel segmentation in the wild. Chinese Automation Congress (cac), 2015, 474–479. https://doi.org/10.1109/CAC.2015.7382547
Lu, H., Cao, Z., Xiao, Y., Li, Y., & Zhu, Y. (2016). Region-based colour modelling for joint crop and maize tassel segmentation. Biosystems Engineering, 147, 139–150. https://doi.org/10.1016/j.biosystemseng.2016.04.007
Madec, S., Jin, X., Lu, H., De Solan, B., Liu, S., Duyme, F., et al. (2019). Ear density estimation from high resolution RGB imagery using deep learning technique. Agricultural and Forest Meteorology, 264, 225–234. https://doi.org/10.1016/j.agrformet.2018.10.013
Marcial-Pablo, M. D. J. U., Gonzalez-Sanchez, A., Jimenez-Jimenez, S. I. A. N., Ontiveros-Capurata, R. E., & Ojeda-Bustamante, W. (2019). Estimation of vegetation fraction using RGB and multispectral images from UAV. International Journal of Remote Sensing, 40(2), 420–438. https://doi.org/10.1080/01431161.2018.1528017
Moreira, J. N., Silva, P. S. L., Silva, K. M. B., Dombroski, J. L. D., & Castro, R. S. (2010). Effect of detasseling on baby corn, green ear and grain yield of two maize hybrids. Horticultura Brasileira, 28(4), 406–411. https://doi.org/10.1590/S0102-05362010000400005
Niu, Q., Feng, H., Yang, G., Li, C., Yang, H., Xu, B., & Zhao, Y. (2018). UAV-based digital imagery for monitoring plant height and leaf area index of maize breeding material. Transactions of the Chinese Society of Agricultural Engineering, 34(5), 73–82. https://doi.org/10.11975/j.issn.1002-6819.2018.05.010
Niu, Y., Zhang, L., Zhang, H., Han, W., & Peng, X. (2019). Estimating above-ground biomass of maize using features derived from UAV-based RGB imagery. Remote Sensing, 11(11), 1261. https://doi.org/10.3390/rs11111261
Ortega-Blu, R., & Molina-Roco, M. (2016). Evaluation of vegetation indices and apparent soil electrical conductivity for site-specific vineyard management in Chile. Precision Agriculture, 17(4), 434–450. https://doi.org/10.1007/s11119-016-9429-x
Petrich, L., Lohrmann, G., Neumann, M., Martin, F., Frey, A., Stoll, A., et al. (2020). Detection of Colchicum autumnale in drone images, using a machine-learning approach. Precision Agriculture, 21(6), 1291–1303. https://doi.org/10.1007/s11119-020-09721-7
Ranum, P., Pena-Rosas, J. P., & Garcia-Casal, M. N. (2014). Global maize production, utilization, and consumption. Annals of the New York Academy of Sciences, 1312(1), 105–112. https://doi.org/10.1111/nyas.12396
Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234–241). Springer. https://doi.org/10.1007/978-3-319-24574-4_28
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4510–4520). https://doi.org/10.1109/CVPR.2018.00474
Soontranon, N., Srestasathiern, P., & Rakwatin, P. (2014). Rice growing stage monitoring in small-scale region using ExG vegetation index. In 2014 11th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications And Information Technology (Ecti-Con), 1–5. https://doi.org/10.1109/ECTICon.2014.6839830
Tan, C., Samanta, A., Jin, X., Tong, L., Ma, C., Guo, W., et al. (2013). Using hyperspectral vegetation indices to estimate the fraction of photosynthetically active radiation absorbed by corn canopies. International Journal of Remote Sensing, 34(24), 8789–8802. https://doi.org/10.1080/01431161.2013.853143
Tang, W., Zhang, Y., Zhang, D., Yang, W., & Li, M. (2011, November). Corn tassel detection based on image processing. In 2012 international workshop on image processing and optical engineering (Vol. 8335, p. 83350J). International Society for Optics and Photonics. https://doi.org/10.1117/12.917672
Toda, Y., Okura, F., Ito, J., Okada, S., Kinoshita, T., Tsuji, H., et al. (2020). Training instance segmentation neural network with synthetic datasets for crop seed phenotyping. Communications Biology, 3(1), 173. https://doi.org/10.1038/s42003-020-0905-5
Tu, S., Pang, J., Liu, H., Zhuang, N., Chen, Y., Zheng, C., et al. (2020). Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images. Precision Agriculture, 21(5), 1072–1091. https://doi.org/10.1007/s11119-020-09709-3
Tucker, C. J. (1979). Red and photographic infrared linear combinations for monitoring vegetation. Remote Sensing of Environment, 8(2), 127–150. https://doi.org/10.1016/0034-4257(79)90013-0
Wang, X., Zhang, R., Song, W., Han, L., Liu, X., Sun, X., et al. (2019). Dynamic plant height QTL revealed in maize through remote sensing phenotyping using a high-throughput unmanned aerial vehicle (UAV). Scientific Reports, 9(1), 3458. https://doi.org/10.1038/s41598-019-39448-z
Westgate, M. E., Lizaso, J., & Batchelor, W. (2003). Quantitative relationships between pollen shed density and grain yield in maize. Crop Science, 43(3), 934–942. https://doi.org/10.2135/cropsci2003.0934
Winterhalter, L., Mistele, B., & Schmidhalter, U. (2012). Assessing the vertical footprint of reflectance measurements to characterize nitrogen uptake and biomass distribution in maize canopies. Field Crops Research, 129, 14–20. https://doi.org/10.1016/j.fcr.2012.01.007
Yan, G., Li, L., Coy, A., Mu, X., Chen, S., Xie, D., et al. (2019). Improving the estimation of fractional vegetation cover from UAV RGB imagery by colour unmixing. ISPRS Journal of Photogrammetry and Remote Sensing, 158, 23–34. https://doi.org/10.1016/j.isprsjprs.2019.09.017
Ye, H., Huang, W., Huang, S., Wu, B., Dong, Y., & Cui, B. (2018). Remote estimation of nitrogen vertical distribution by consideration of maize geometry characteristics. Remote Sensing, 10(12), 1995. https://doi.org/10.3390/rs10121995
Zabawa, L., Kicherer, A., Klingbeil, L., Töpfer, R., Kuhlmann, H., & Roscher, R. (2020). Counting of grapevine berries in images via semantic segmentation using convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 164, 73–83. https://doi.org/10.1016/j.isprsjprs.2020.04.002
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017). Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2881–2890). https://doi.org/10.1109/CVPR.2017.660
Acknowledgements
This research was supported by the National Natural Science Foundation of China (Grant No. 42071426), and Central Public‐interest Scientific Institution Basal Research Fund for Chinese Academy of Agricultural Sciences (Grant Nos. Y2020YJ07, S2018QY01).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shao, M., Nie, C., Cheng, M. et al. Quantifying effect of tassels on near-ground maize canopy RGB images using deep learning segmentation algorithm. Precision Agric 23, 400–418 (2022). https://doi.org/10.1007/s11119-021-09842-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11119-021-09842-7