Development of a LiDAR-guided section-based tree canopy density measurement system for precision spray applications

doi:10.1016/j.compag.2021.106053

Computers and Electronics in Agriculture

Volume 182, March 2021, 106053

https://doi.org/10.1016/j.compag.2021.106053 Get rights and content

Highlights

•
A 3D tree canopy characterization system was integrated for precision spraying.
•
Segmentation of tree trunk, trellis wires, and support poles was performed.
•
A correlation of 0.95 (R² = 89.30%) between points and leaves was achieved with Fuji variety.
•
Canopy volume showed a strong correction of 0.98 (R² = 95.90%) to leave number.
•
Canopy density map was generated to visualize leaves distribution.

Abstract

An unmanned ground-based canopy density measurement system to support precision spraying in apple orchards was developed to precisely apply pesticides to orchard canopies. The automated measurement system was comprised of a light detection and ranging (LiDAR) sensor, an interface box for data transmission, and a laptop computer. A data processing and analysis algorithm was developed to measure point cloud indices from the LiDAR sensor to describe the distribution of tree canopy density within four sections according to the position of the trellis wires. Experiments were conducted in two orchard sites, one with GoldRush (larger trees) and the other one with Fuji (smaller trees) apple trees. Tree leaves were counted manually from each section separated by trellis wires. Field evaluation results showed a strong correlation of 0.95 (R² = 89.30%) between point cloud data and number of leaves for the Fuji block and a correlation of 0.82 (R² = 67.16%) was obtained for the GoldRush block. A strong correlation of 0.98 (R² = 95.90%) was achieved in the relationship between canopy volume and number of leaves. Finally, a canopy density map was generated to provide a graphical view of the tree canopy density in different sections. Since accurate canopy density information was computed, it is anticipated that the developed prototype system can guide the sprayer unit for reducing excessive pesticide use in orchards.

Introduction

Conventional agriculture relies heavily on a high-level use of plant protection products, commonly known as pesticides. Pesticides play a critical role in increasing crop quality and productivity. Oerke et al. (2012) suggested that failure to use plant protection products against insects, diseases, pests, and weeds could result in up to 65% of crop yield losses. On the contrary, pesticide misuse represents a serious concern about their adverse impact on the non-targets, including humans, environment, and ecosystems (Alavanja et al., 1996, Deveau, 2009). Additionally, pesticide use caused about $8.2 billion in annual environmental and economic losses in the United States (Pimentel & Burgess, 2014). To address this concern, the reduction of plant protection products is very important and crucial when considering agricultural sustainability and profitability.

Current development of innovative management strategies has shown a significant reduction of pesticides and improves efficacy and safety by adopting the modern breakthrough in electronics (Ampatzidis et al., 2018). Precision spraying is one of the modern crops management strategies that assist management decisions (e.g. spraying) according to estimated variability in the field, aiming to reduce agricultural inputs. Precision spraying strategies have been utilized by researchers in the recent few decades for site-specific managements including weeds (Hunter et al., 2020), diseases (Yang, 2020), and pests (Zhong et al., 2018). The core concept of precision spraying is to adjust the spray volume by controlling nozzle flow rate.

Adjustment of the spray deposits according to the tree canopy characteristics offers the chance of decreasing pesticide use and environmental contamination (Nan et al., 2019). The tree canopy foliage plays an important role in determining the amount of spray volume required in an individual tree. However, tree canopies are not uniform in terms of density and volume. The variability of canopy foliage density is shown in Fig. 1. Identifying tree canopy foliage density can characterize the tree structure and determine the appropriate spray volume for precise pesticide applications (Chen et al., 2012, Hu and Whitty, 2019, Wei and Salyani, 2005). It also helps to adjust the pesticide application rate, spray flow rate, and air supply volume to better manage the orchards while spraying (Gil et al., 2007, Jeon and Zhu, 2012, Llorens et al., 2010, Shen et al., 2017).

A range of techniques used for measuring tree canopy foliage density characteristics has included visible-range camera sensor (Asaei et al., 2019), ultrasonic sensor (Gil et al., 2007), spectral sensor, infrared sensor (He et al., 2011), and laser sensor (Chen et al., 2012, Liu and Zhu, 2016). Despite the considerable efforts reflected in characterizing tree canopies, challenges still exist to accurately implement the developed strategies in real-time field conditions, due to uncontrollable weather conditions and system limitations. The performance of the precision management system is significantly reduced in the field conditions due to illumination variations, wind speed and direction, and system vibrations when a camera-based sensing system is used (Asaei et al., 2019). Ultrasonic based sensing systems provide inconsistent data due to the large angle of divergence of ultrasonic waves and uncontrollable environmental conditions in fields (Zhang et al., 2018). Similarly, studies have shown the difficulty of recording accurate data using spectral and infrared sensors due to high sensitivity to the outdoor field illumination and weather conditions (Zhang et al., 2018). Conversely, laser-based sensing techniques are not affected by the outdoor field weather conditions and provide more accurate detection results (Liu & Zhu, 2016).

LiDAR (light detection and ranging) sensing is an active laser scanner-based remote sensing technique applied widely for tree canopy characterizations (Brandtberg et al., 2003, Holmgren and Persson, 2004, Hosoi and Omasa, 2006, Omasa et al., 2007). The LiDAR sensor emits an electromagnetic signal that can bounce off of the vegetation canopy enabling a view of the exterior structure and three-dimensional information of the tree. Calculation of tree canopy foliage density characteristics using a LiDAR sensor have been reported (Auat Cheein et al., 2015, Berk et al., 2020, Chakraborty et al., 2019, Hu and Whitty, 2019). Auat Cheein et al. (2015) estimated three-dimensional structure of orchard trees; in particular, real-time measurement of canopy volume and shape using a LiDAR sensor and computational geometry analysis. Results reported that the accuracy was decreased up to 30%. Underwood et al. (2016) measured the tee canopy volume using terrestrial LiDAR scanner and achieved coefficient of determination (R² = 0.77) for establishing the relationship between canopy volume and yield. Chakraborty et al. (2019) used a mobile 3D LiDAR mapping system to measure canopy volume for apple trees and grapevines. They reported correlation values of 0.81 and 0.51 between manual and automatic measurements using Convex hull and Voxel grid methods, respectively. However, the Voxel grid method is computationally intense and is affected by the voxel size. The Convex hull method showed inferior performance compared to the Voxel grid method. Hu and Whitty (2019) evaluated a tree canopy density mapping system for a trellis-structured apple orchard where all points generated from an individual tree were included. However, a trellis-structured apple orchard may have many points produced by the wire-plane and also from the main tree trunk that need to be removed before canopy density calculation. Berk et al. (2020) established a relationship by conducting laboratory experiments for measuring tree leaf area, but low accuracies were reported. Among the studies surveyed, most researchers have tried to measure tree canopy density based on the volume of individual trees considering all points; however, the density of the whole tree cannot precisely guide the sprayer unit because the precision sprayer may have multiple nozzles on each side which need to be controlled separately. Section-based canopy density measurement leads to assessment of foliage density by dividing the tree into sections (e.g., bottom, middle, and top, etc.). The computed density information of the canopy sections can separately guide/control the corresponding nozzle facing each section. Since the precision spraying system requires nozzle flow rates to be continuously controlled during orchard spraying, the spray decision input, i.e., section-based tree canopy density information needs to be measured automatically.

The primary goal of this study was to develop an automated section-based tree canopy density measurement system for precise pesticide spraying using a LiDAR sensor. The specific objectives were to: (i) establish a relationship between the point cloud and canopy foliage without considering trellis-wires, support poles, and tree trunk (ii) predict the number of leaves in each section with the density measurement algorithm (iii) measure the tree canopy volume and generate the canopy density map for providing guideline information for variable-rate spraying.

In this study, we performed three major tasks based on LiDAR scanned data for tree canopy density and volume measurements: point cloud data acquisition, tree canopy points segmentation, and canopy density and volume measurements. Data were acquired through LiDAR integrated sensing system. A sample consensus algorithm was used to remove the ground points. Unnecessary points from tree trunks, trellis wires, and support poles were removed using a processing algorithm aimed to segment only canopy points. The canopy density and volume were measured, and canopy density map was generated. The canopy density map generated in this work provides a graphical view of tree leaves distributions in different sections, which can be used later for spraying operation in the orchards.

Section snippets

Test orchards

Two orchard sites with GoldRush (site 1: 39°56′15.8″N, 77°15′21.0″W) and Fuji (site 2: 39°56′19.1″N, 77°15′17.5″W) apple varieties, located at Penn State Fruit Research and Extension Center (FREC), Biglerville, PA, USA, were used (Fig. 2). Both orchards use a trellis system to support trees, including three tiers of trellis wires and support poles. For the GoldRush block, the trees were trained as a tall spindle structure. The trees were planted in 2009 with an inter-row spacing of 6.10 m and

Canopy identification in sections

The point cloud data of five consequent trees in each orchard site were processed using the developed algorithms. Fig. 12, Fig. 13 show the tree canopies in the two test sites with and without trellis wire, poles, and tree trunks. These points were marked in different colors to represent four sections of tree canopy divided by the trellis wires.

Fig. 12, Fig. 13(a) show the point cloud data with points removed for the ground and neighboring rows. The trellis wires, poles, and tree trunks were

Conclusions

Apple tree canopy density was assessed using a ground-based LiDAR guided sensing system. Point cloud data were acquired from two orchard sites with GoldRush and Fuji apple varieties. A processing algorithm was scripted in the MATLAB® programming environment. Experiments were conducted with T-trellis structured orchards; therefore, tree trunk, trellis wires, and support poles were extracted to separate only the canopy points from the acquired data. Apple leaves were counted manually by a

CRediT authorship contribution statement

Md Sultan Mahmud: Conceptualization, Investigation, Methodology, Validation, Writing - original draft. Azlan Zahid: Investigation, Methodology, Validation. Long He: Conceptualization, Supervision, Writing - review & editing, Funding acquisition. Daeun Choi: Supervision, Writing - review & editing. Grzegorz Krawczyk: Supervision, Writing - review & editing. Heping Zhu: Supervision, Writing - review & editing. Paul Heinemann: Supervision, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This study was supported in part by United States Department of Agriculture (USDA)’s National Institute of Food and Agriculture (NIFA) Federal Appropriations under Project PEN04653 and Accession No. 1016510, a USDA NIFA Crop Protection and Pest Management Program (CPPM) competitive grant (Award No. 2019-70006-30440), and a Northeast Sustainable Agriculture Research and Education (SARE) Graduate Student Grant GNE20-234-34268. The authors would like to give special thanks to Xiaohu Jiang

References (36)

H. Asaei et al.
Site-specific orchard sprayer equipped with machine vision for chemical usage management
Comput. Electron. Agric.
(2019)
F.A. Auat Cheein et al.
Real-time approaches for characterization of fully and partially scanned canopies in groves
Comput. Electron. Agric.
(2015)
P. Berk et al.
Digital evaluation of leaf area of an individual tree canopy in the apple orchard using the lidar measurement system
Comput. Electron. Agric.
(2020)
T. Brandtberg et al.
Detection and analysis of individual leaf-off tree crowns in small footprint, high sampling density lidar data from the eastern deciduous forest in North America
Remote Sens. Environ.
(2003)
M. Chakraborty et al.
Evaluation of mobile 3D light detection and ranging based canopy mapping system for tree fruit crops
Comput. Electron. Agric.
(2019)
E. Gil et al.
Variable rate application of plant protection products in vineyard using ultrasonic sensors
Crop Prot.
(2007)
J. Holmgren et al.
Identifying species of individual trees using airborne laser scanner
Remote Sens. Environ.
(2004)
M. Hu et al.
An evaluation of an apple canopy density mapping system for a variable-rate sprayer
IFAC-PapersOnLine
(2019)
J. Llorens et al.
Variable rate dosing in precision viticulture: use of electronic devices to improve application efficiency
Crop Prot.
(2010)
Y. Nan et al.
Estimating leaf area density of Osmanthus trees using ultrasonic sensing
Biosyst. Eng.
(2019)

T. Palleja et al.

Sensitivity of tree volume measurement to trajectory errors from a terrestrial lidar scanner

Agric. For. Meteorol.

(2010)

J.P. Underwood et al.

Mapping almond orchard canopy volume, flowers, fruit and yield using lidar and vision sensors

Comput. Electron. Agric.

(2016)

T. Vidal et al.

Reduction of fungal disease spread in cultivar mixtures: Impact of canopy architecture on rain-splash dispersal and on crop microclimate

Agric. For. Meteorol.

(2017)

C. Yang

Remote sensing and precision agriculture technologies for crop disease detection and management with a practical application example

Engineering

(2020)

L. Zeng et al.

Semantic segmentation of sparse 3D point cloud based on geometrical features for trellis-structured apple orchard

Biosyst. Eng.

(2020)

M.C. Alavanja et al.

The agricultural health study

Environ. Health Perspect.

(1996)

Y. Ampatzidis et al.

Voice-controlled and wireless solid set canopy delivery (VCW-SSCD) system for mist-cooling

Sustainability (Switzerland)

(2018)

Y. Chen et al.

Development of a variable-rate sprayer with laser scanning sensor to synchronize spray outputs to tree structures

Trans. ASABE

(2012)

Cited by (41)

Detecting, mapping and digitising canopy geometry, fruit number and peel colour in pear trees with different architecture
2024, Scientia Horticulturae
Increasing farm labour and input costs and the requirement for more orchard data are leading to rapid advances in technology to improve management systems in fruit production. The study aimed to (i) validate a sensorised platform to estimate fruit number, peel colour and blush coverage in pear orchards with different pear selections and tree architectures, (ii) establish relationships between fruit number, peel colour and canopy geometry features, and (iii) evaluate the platform for mapping and digitising orchard features. The study was carried out over two years in an experimental ‘ANP-0131′ orchard and in one year in two commercial pear orchards (‘ANP-0131′ and ‘PremP009’). Predictions of fruit number and blush coverage were compared in traditional three-dimensional (3D) and modern high-density two-dimensional (2D) training systems. Overall, prediction errors for fruit number were < 6.5 % in all the training systems, but improved performance was achieved in vertical 2D configurations (% standard errors = 2.2 %). Fruit number and blush coverage per unit of leaf area were higher in 2D compared to 3D training systems. Blush coverage predictions in ‘ANP-0131′ were reliable (R² = 0.67, RMSE = 3.70 %). Accurate predictions of blush coverage classes were achieved by modifying sample variance. Fruit number and blush coverage were negatively affected by increasing canopy size. The platform proved useful for mapping and digitisation. Spatial heatmaps of orchard features provided a valuable visual aid to identify zones for priority interventions for peel colour enhancement.
Force and motion behaviour of crop leaves during spraying
2023, Biosystems Engineering
In agricultural spraying, the force characteristics and motion behaviour of plant leaves excited by spray clouds is important for improving pesticide efficacy. In this study, the forces and motion of Capsicum leaves during excitation by spray were simultaneously recorded using a tailor-made 3D force transducer and high-speed video system. The results showed that the leaf forces during spraying could be divided into three stages: a rising stage, a steady but fluctuation stage, and a constant value stage, with peak forces varying between 0.25 and 0.52 mN cm⁻² occurring within 6–10 s. To further investigate the forces on a leaf, a six-parameter droplet cluster loading model was developed. The model accounts for both the physical parameters of the leaf and the fluctuations induced by the dynamic components of the spray. Determining model parameters was accomplished using a global parameter identification methodology enhancing the congruence between the physical model and the measured data. Under the impact of a droplet cluster, the leaf persistently demonstrates a downward shaking motion accompanied by substantial bending deformation. The maximum motion velocity recorded was 0.93 m s⁻¹, with vibration frequencies ranging between 2.58 Hz and 7.75 Hz. The findings can help understand the interactions between droplets and leaves and contribute valuable knowledge towards optimising pesticide application.
Vision-based normalized canopy area estimation for variable nitrogen application in apple orchards
2023, Smart Agricultural Technology
Normalized canopy area is a measure of spatial canopy fill or canopy coverage in a tree. It is an important factor for growers to consider in making various farming decisions, such as timing and amount of fertilization, pesticide application, and irrigation, as it gives a sense of the overall canopy growth. This study focused on estimating the normalized canopy area of apple trees using a ground-based stereo-vision system in a commercial orchard under natural lighting conditions. A color, depth, and height threshold, followed by a point density-based outlier removal was applied to segment the target tree canopies. The segmented canopy area was compared with the area estimated with manual segmentation, which showed an F1 score of 0.78. The overall growth of the tree canopies was then represented using normalized canopy area, which was calculated as a fraction of pixels occupied by the tree foliage over the total possible area a tree could occupy for a given tree spacing. It was found that the normalized canopy area of individual trees was highly correlated with the surface volume of the canopy (r = 0.7), which is a traditionally used parameter to assess or represent canopy growth. The estimated normalized canopy area was then compared against experts' nitrogen recommendation for the sample trees, which was based on their visual assessment of the vigor/growth of the trees. Four individual experts (one apple farmer, one orchard manager, and two horticulturists with research and outreach experience) were asked to provide their independent nitrogen recommendation and an average recommendation level was calculated. The comparison showed an overall correlation coefficient of -0.5, indicating an opposite relationship (higher the canopy density, lower the need for nitrogen and vice versa) between the normalized canopy area and the average of the experts' nitrogen recommendation. When the relationship between the normalized area and nitrogen recommendation level was assessed for individual experts, the correlation coefficient was estimated to be -0.86, -0.84, -0.96, and -0.78, respectively, for Expert1, Expert2, Expert3, and Expert4 respectively. The results from this study identified a critical measure for assessing nitrogen needs at individual tree levels in apple orchards and provides an automated tool to estimate the measure. The outcomes of this study could be used as a part of an integrated decision support system for orchard operations (e.g., fertilization or irrigation) that may include additional canopy characteristics such as spectral signature, trunk diameter, and rate of change of leaf color in the fall.
Effects of laser scanner quality and tractor speed to characterise apple tree canopies
2023, Smart Agricultural Technology
Precision orchard management requires non-destructive sensor technologies to identify key sources of variation in apple yield and quality so operations such as fruit thinning, root-pruning as well as harvesting can be better targeted and optimised. We compared the data quality of two light detection and ranging (LiDAR) scanners to assess their ability to estimate the volume of ‘Braeburn’ apple trees. Scanner one was a high-quality tractor-mounted LiDAR (SICK, LDMRS 400001, Düsseldorf, Germany) and the second system consisted of two lower-cost SICK LiDAR sensors fitted on a commercial orchard sprayer. To compare the two scanner systems, two tree row sections with different growth vigour were created by targeted root pruning to reduce annual shoot growth. LiDAR scans and reference assessments were made at the end of the growing season. In addition, LiDAR data were obtained at two tractor speeds: 1.5 and 7.0 km/h (standard spraying speed). Results show key canopy features like the beginning/end of rows or large gaps can be detected with the lower-cost sensors. However, tractor speed had a major effect on data quality. At 1.5 km/h, alpha-shaped tree volumes of the high-quality LiDAR show a high correlation (R² = 0.84) with the reference assessments, whereas the data quality of the lower-cost sensors was not high enough to represent the tree volume with the alpha-shape algorithm (R² = 0.09). Moreover, this work raises the question of how precise a sensor-based tree volume needs to be determined to describe heterogeneity in an orchard. Specifically, how to best match sensor data to manual methods of canopy measurement and allow sensor data to be used for multiple applications.
Unmanned aerial vehicle based tree canopy characteristics measurement for precision spray applications
2023, Smart Agricultural Technology
Citation Excerpt :
An alpha shape algorithm (Dylan [26]) was used with three alpha values (0.35, 0.40, and 0.45) to calculate individual tree volume (Fig. 2). The canopy point clouds preprocessing and processing steps were described in detail by Mahmud et al. [9], while it did not measure any ground-based canopy coverage due to the complexity and impracticality compared to aerial measurement. Images were acquired using a rotary-wing unmanned aerial vehicle (UAV) DJI Matrice 200 (DJI Technology Inc., Shenzhen, China) with a high-resolution RGB Zenmuse X5S camera (DJI Technology Inc., Shenzhen, China) on June 21, 2021 (Fig. 3).
The critical components for applying the correct amount of agrochemicals are fruit tree characteristics such as canopy height, canopy volume, and canopy coverage. An unmanned aerial vehicle (UAV)-based tree canopy characteristics measurement system was developed using image processing approaches. The UAV captured images using a high-resolution red-green-blue (RGB) camera. A digital surface model (DSM) and a digital terrain model (DTM) were generated from the captured images. A tree canopy height map was generated from the subtraction of DSM and DTM. A total of 24 apple trees were randomly targeted to measure the canopy characteristics. Region of interest (ROI) was generated across the boundary of each targeted tree. The height of all pixels within each ROI was computed separately. The pixel with maximum height was considered as the height of the respective tree. For computing canopy volume, the sum of all pixel heights from individual ROI was multiplied by the square of ground sample distance (GSD) of 5.69 mm·pixel⁻¹. A segmentation method was employed to calculate the canopy coverage of the individual trees. The segmented canopy pixel area was divided by the total pixel area within the ROI. The results showed an average relative error of 0.2 m(6.64%) while comparing automatically measured tree height with ground measurements. For tree canopy volume, a mean absolute error of 0.25 m³ and a root mean square error of 0.33 m³ were achieved. The study estimated the possible agrochemical requirement for spraying the fruit trees, ranging from 0.1 to 0.32 l based on tree canopy volumes. The overall investigations suggest that the UAV-based tree canopy characteristics measurements could be a potential tool to calculate the pesticide requirement for precision spraying applications in tree fruit orchards.
Real-time detection of street tree crowns using mobile laser scanning based on pointwise classification
2023, Biosystems Engineering
For targeted spraying in urban streets using mobile laser scanning (MLS), it is crucial to continuously detect tree crowns in real time from MLS data collected online. Crown detection based on a pointwise classification method, which predicts whether a point belongs to a crown using a binary classifier composed of a set of local features extracted from the neighbourhood of the point, is suitable for online data processing. However, the computational complexity of pointwise detection methods needs to be reduced to meet the real-time requirements of online targeted spraying. This paper proposes a real-time detection method for street tree crowns using MLS based on pointwise classification. A cubic neighbourhood is adopted from which a set of statistical features are extracted. A coarse-to-fine neighbourhood search method based on an MLS data grid storage structure is proposed to quickly query the neighbours. The classification performance and computational time of single features are evaluated. Then, the features are ranked using the technique for order preference by similarity to the ideal solution (TOPSIS). Finally, a subset of features that are easy to calculate and have high discrimination are selected and fused into a crown detector using a random forest algorithm. During online detection, the point cloud is sampled in terms of the vehicle movement and LiDAR measurement before being detected. An MLS point cloud collected from a 451 m urban street is used in the experiments. When the neighbourhood size is set to 0.3 m, the detection time per scanline is 24.84 ms on the unsampled test set with an $F_{1}$ score of 0.9927. When the test set is sampled with a total sampling rate of 40, the detection time is reduced to 1% with an $F_{1}$ score of 0.9906. The experimental results show that the proposed method can provide real-time and accurate detection of street tree crowns for online targeted spraying.

View all citing articles on Scopus

View full text

Original papersDevelopment of a LiDAR-guided section-based tree canopy density measurement system for precision spray applications

Highlights

Abstract

Introduction

Section snippets

Test orchards

Canopy identification in sections

Conclusions

CRediT authorship contribution statement

Declaration of Competing Interest

Acknowledgement

Comput. Electron. Agric.

Comput. Electron. Agric.

Comput. Electron. Agric.

Remote Sens. Environ.

Comput. Electron. Agric.

Crop Prot.

Remote Sens. Environ.

IFAC-PapersOnLine

Crop Prot.

Biosyst. Eng.

Agric. For. Meteorol.

Comput. Electron. Agric.

Agric. For. Meteorol.

Engineering

Biosyst. Eng.

The agricultural health study

Environ. Health Perspect.

Voice-controlled and wireless solid set canopy delivery (VCW-SSCD) system for mist-cooling

Sustainability (Switzerland)

Development of a variable-rate sprayer with laser scanning sensor to synchronize spray outputs to tree structures

Trans. ASABE

Original papers
Development of a LiDAR-guided section-based tree canopy density measurement system for precision spray applications