Assessment of grass lodging using texture and canopy height distribution features derived from UAV visual-band images
Introduction
Grass is used extensively in temperate pastures as a high-quality source for feeding animals or as turf for home lawns, golf courses and recreational areas (Abel et al., 2017). Grassland is estimated to cover approximately 72 million hectares within the European Union with grass seed production estimated at a total value of 23 billion Euro (ESA, 2016). Seed yield is a key trait for all forage and turf grass species (Boelt and Studer, 2010). Lodging, which is the permanent displacement of a crop stalk from its natural upright position and is due to internal and external factors, is a major limiting factor for the yield, quality and harvesting efficiency of crops worldwide (Berry and Spink, 2012; Griffith, 2000; Kendall et al., 2017; Niu et al., 2012).
Several studies have been performed to analyse the mechanism and the cause of crop lodging (Brune et al., 2018; Xue et al., 2017; Baker et al., 1998), which have presented quantitative measures to evaluate lodging resistance of individual crops. However, it is often of interest to assess the severity of lodging and to map the seasonal lodging risk in an entire crop field; thus, monitoring and assessing lodging severity occupy an important role in the research of crop lodging. The traditional method to identify the lodging area and lodging severity relies on manual field assessments, which are labour intensive, time consuming and subjective. When large and entire fields are involved, manual inspection is infeasible. Thus, one possible solution is to use image analysis to give an objective assessment of the field and to use remote sensing technique to capture the images in order to reduce the overall labour and time.
Static digital images obtained from ground-based devices have been used to assess crop lodging. Ogden et al. (2002) proposed a functional regression assessment metric to predict the lodging severity based on images acquired at a 3-metre height nadir-view over a rice field. Masuda et al. (2013) used variances of pixel values in the subbands acquired by wavelet transform as features to judge whether rice plant lodging occurred or not by a threshold method. Zhang et al. (2012) detected grain quality of maize plant under lodging and non-lodging circumstances, using a ground-based spectrometer. In their study, distinguishable spectral features were extracted by utilising the continuous wavelet transform method and the partial least squares regression. To extend usability of these methods at large farm scale, calibration and re-evaluation are required.
Large areas can quickly and easily be scanned using different sensor technologies because of the rapid development of remote sensing. Combined with image analysis, this technology may provide efficient and reliable tools for obtaining timely grass lodging information over a large area of the field. Sensors can acquire data remotely while being on board different platforms, such as satellites, airplanes and unoccupied aerial vehicles (UAVs) (Kasampalis et al., 2018). Radar-based satellite data have also been reported as a method to detect lodging. Yang et al. (2015) investigated using RADARSAT-2 radar satellite imagery and found that polarimetric features, such as the backscattering intensities and scattering components from polarimetric decompositions, were sensitive to the lodging of wheat. However, their study did not provide quantitative information on lodging monitoring. Chen et al. (2016) found similar results in sugarcane lodging detection. Han et al. (2018a) found a correlation between a Sentinel-1 radar polarisation index and corn height and used it to detect corn lodging. Unfortunately, backscatter and polarimetric features are also related to crop growth, senescence and density; hence, these methods can only be applied at certain growth stages.
Compared to satellite and airplane, a UAV with proper sensors offers a flexible, convenient and cost-effective way to provide desired and high-resolution images of crop fields. Previous work on lodging assessment using UAVs have studied colour, texture and height derived from RGB images as well as temperature. Rajapaksa et al. (2018) used grey-level co-occurrence matrix texture features and support vector machine (SVM) to show the presence and absence of lodging in wheat and canola at a subplot scale. Yang et al. (2017) constructed a pixel-wise classifier to classify rice as lodging or non-lodging using RGB colour, texture and digital surface model (DSM) extracted from UAV images. Han et al. (2018b) proposed a method using similar features and two nomogram models to predict the probability of maize lodging and to identify the protective factors and risk factors related to maize lodging. However, the stability of the predicted model needed further study due to lack of data validation for multiple growing stages. Chu et al. (2017) used the UAV-estimated maize height to assess maize lodging severity. This proposed method segmented the individual maize rows into multiple grid cells and determined the lodging severity on a per-plant basis based on the height percentiles against pre-set thresholds within individual grid cells. However, threshold tuning was inevitably required for adapting this method to a variety of plant types over different growth stages. Similarly, canopy height measurement by small-balloon photogrammetry was employed to evaluate lodging rate in a buckwheat area (Murakami et al., 2012). More recently, Liu et al. (2018) found that temperature was distinct between lodging and non-lodging rice areas, and an SVM lodging recognition model using colour, texture and temperature indices was established. The temperature indices were collected using a handheld platform while the colour and texture indices were collected using a UAV.
Although much progress in the assessment and prediction of crop lodging has been made, we suggest some areas that can be further improved. The previous studies primarily perform binary classification into lodging or not lodging. In pixel-wise classification, this may lead to the “salt and pepper” effect, where single pixels in a larger area are misclassified (Orynbaikyzy et al., 2019). This effect may be circumvented by grouping pixels before classifying them jointly. When classifying groups of pixels, the classification problem is, however, no longer necessarily binary, since some pixels may correspond to lodging and others not. A second area is utilisation of a standard lodging scale. Currently there are no standard reference scale to represent lodging in a more detailed way (e.g. mild, moderate and severe), and we suggest a lodging scale that can be utilised worldwide in different crops.
To overcome these limitations, features that are different from previous studies should be explored and adopted. Features used alone or combined should be sensitive to different lodging severity. Furthermore, a method to measure and to validate different lodging severity of small areas, such as individual plots, should be adopted to avoid the shortcoming of pixel-wise classification. In extension to this, a non-binary lodging classification should be developed (e.g., mild, moderate and severe lodging). In the event of lodging, the crop canopy structure is destroyed so that the stem is inclined at a certain angle and the plant height is reduced. As a result, both texture and the crop height distribution within the canopy change. Histograms of oriented gradient (HOG) is a feature representing an image with a set of local histograms counting the occurrences of gradient orientations within a local image cell. It was successfully applied for pedestrian detection by Dalal and Triggs (2005), and it was found that the HOG features significantly outperformed existing feature sets for human detection. Meanwhile, crop height has been widely used in establishing the crop yield models that translate the relationship between crops and their environment (Borra-Serrano et al., 2019; Caruso et al., 2019; Hassan et al., 2019). To our best knowledge, HOG features and crop height distribution have not previously been studied for crop lodging severity assessment.
The objective of this study is to explore an efficient and robust way to assess the lodging severity using four different grass seed crops as model crops. For this purpose, an RGB camera mounted on a UAV is used to capture images of a field plot experiment. Two types of features extracted from the individual plots are investigated: (i) HOG feature and (ii) canopy height distribution. An SVM classifier is used to classify the individual plots into three lodging categories: no lodging, medium lodging and severe lodging.
Section snippets
Field experiments
The present study was part of two comprehensive field plot experiments conducted at two locations at Research Centre Flakkebjerg, Denmark (55o32’ N and 11o39’ E), from 2016 to 2020. The two locations were Bjaerup (Bj) and Mindelundsvej (Min) (Fig. 1). The size of each plot was 2.5 × 8 m2. The soil at Flakkebjerg is classified as a sandy loam (Haplic Luvisol (FAO)/Typic Hapludalf (USDA)) with clay illuviation below the plough layer (23 cm) and sand lenses. The experimental design was a fully
HOG features
The HOG feature parameter grid search was performed by training individual SVM classifiers on the training set and subsequently evaluating the classifiers on the validation set and test set. Cell sizes of 8, 16 and 32 pixels as well as block sizes of 2, 3, 4, 5 and 6 cells were evaluated. The combination of a cell size of 32 and block size of 6 was not evaluated because the effective size of a single block exceeded the plot image size. For each combination of cell and block size, an SVM
Accuracy of UAV images for grass lodging severity assessment
Based on the existing studies, we find that lodging has been studied most extensively in wheat, followed by barley, rice and cereals in general. In regard to the use of remote sensing to detect lodging, the study is still in an early stage (Chauhan et al., 2019). In previous studies, colour, texture, temperature, spectral and crop height information have been explored as feature parameters to detect crop lodging (Liu et al., 2018; Rajapaksa et al., 2018; Wang et al., 2018; Wilke et al., 2019).
Conclusion
In this paper, an efficient and robust method of assessing grass seed crops lodging severity using UAV images was presented. Two distinct features, the HOG and canopy height distribution, were derived from individual plot images. SVM classifiers based on the HOG feature and grass height distribution, both combined and individually, were used to classify the individual plots into three lodging severities, namely no lodging, medium lodging and severe lodging. Different parameters of HOG features
Declaration of Competing Interest
None.
Acknowledgements
We are grateful to the Key-Area Research and Development Program of GuangDong Province under Grants 2019B020214002 and the China Scholarship Council (CSC) for financial support of S.Y. TAN in this work.
References (45)
- et al.
Agroview: cloud-based application to process, analyze and visualize UAV-collected data for precision agriculture applications utilizing artificial intelligence
Comput. Electron. Agric.
(2020) Changes in dry matter, carbohydrate and seed yield resulting from lodging in three temperate grass species
Ann. Bot.
(2000)- et al.
Path and correlation analysis of perennial ryegrass (Lolium perenne L.) seed yield components
J. Agron. Crop Sci.
(2017) - et al.
A method for the assessment of the risk of wheat lodging
J. Theor. Biol.
(1998) - et al.
Estimating biomass of barley using crop surface models (CSMs) derived from UAV-based RGB imaging
Remote Sens.
(2014) - et al.
Predicting yield losses caused by lodging in wheat
F. Crop. Res.
(2012) - et al.
Breeding for grass seed yield
- et al.
Canopy height measurements and non-destructive biomass estimation of Lolium perenne swards using UAV imagery
Grass Forage Sci
(2019) - et al.
Closing the phenotyping gap: High resolution UAV time series for soybean growth analysis provides objective data from field trials
Remote Sens.
(2020) - et al.
A biomechanical model for maize root lodging
Plant Soil
(2018)