Orchard management with small unmanned aerial vehicles: a survey of sensing and analysis approaches

Abstract

Advances in sensor miniaturization are increasing the global popularity of unmanned aerial vehicle (UAV)-based remote sensing applications in many domains of agriculture. Fruit orchards (the source of the fruit industry chain) require site-specific or even individual-tree-specific management throughout the growing season—from flowering, fruitlet development, ripening, and harvest—to tree dormancy. The recent increase in research on deploying UAV in orchard management has yielded new insights but challenges relating to determining the optimal approach (e.g., image-processing methods) are hampering widespread adoption, largely because there is no standard workflow for the application of UAVs in orchard management. This paper provides a comprehensive literature review focused on UAV-based orchard management: the survey includes achievements to date and shortcomings to be addressed. Sensing system architecture focusing on UAVs and sensors is summarized. Then up-to-date applications supported by UAVs in orchard management are described, focusing on the diversity of data-processing techniques, including monitoring efficiency and accuracy. With the goal of identifying the gaps and examining the opportunities for UAV-based orchard management, this study also discusses the performance of emerging technologies and compare similar research providing technical and comprehensive support for the further exploitation of UAVs and a revolution in orchard management.

Introduction

Faced with the challenges of a growing global population, rural poverty, and natural resource management, public awareness about sustainable practices in food production has become prominent. Food productivity, e.g., in fruit crops, and the economics of cultivation and irrigated agriculture are important topics in precision agriculture that relate to these challenges. In the case of fruit orchard management, the challenges also concern processes. Within the context of precision agriculture, site-specific management is key to reducing environmental impact and enhancing agricultural economics. Such management includes the general stages of collecting data, mapping orchard variability, and making decisions (Zhang & Kovacs, 2012) and enables each aspect of crop cultivation (e.g., physical, biological, chemical factors) to be understood.

Throughout the growing cycle of fruit crops, a variety of management activities are employed (Fig. 1). Different activities focus on the management of fruit trees at specific growth stages, and some activities are not limited to one growth stage. For instance, thinning generally occurs during the flowering and fruitlet stages. The management activities attracting most research attention have been fruit thinning, fruit-tree pruning, and the assessment of irrigation strategy. Apple-flower number strongly correlates with the final yield (Aggelopoulou et al. 2009). Improving fruit size and quality will bring more economic benefits because of the higher market prices (Bound, 2018). However, without precise management, an excessive number of flowers per tree might lead to poor fruit quality and size. Overload of fruits also threatens the trees (Dennis, 2000). For example, only 7% of apple flowers are necessary for final harvest, which means that flower thinning should be conducted regularly (Greene and Costa 2013). In some cases, manual thinning represents 31% of all cultural costs (costs of activities carried out during an entire fruit-tree growing cycle) (Glozer & Hasey, 2006). Though mechanical thinning effectively reduces labor input, the bottleneck is the precise estimation related to flower intensity. Changes in fruit-tree structural properties can be used to monitor tree crop growth status, and site-specific treatments with fungicides and water can be implemented with the aid of a spatial structural variation map (Mu et al. 2018). Fruit tree physical structure, such as canopy volume, affects fruit quality and yield by intercepting light (Rom, 1991). Trees in fruit orchards are therefore pruned at certain fruit-growing stages, following a pruning strategy. Geometric characteristics like canopy area are the main way of assessing the pruning effects. By measuring this parameter, optimal pruning type and intensity can be selected and applied, to help guarantee the final income of local growers (Castillo-Ruiz et al. 2015; Miranda-Fuentes et al. 2015). In addition, because of the global water shortage, agricultural irrigation is a relevant topic. In the context of deficit irrigation, the water inputs for orchards should meet the realistic demands of individual fruit trees. Mild water stress is often employed to maintain or improve fruit quality and yield in orchards. Thus, to optimize water resources usage, appropriate monitoring is necessary. In summary, projecting into the future, precision orchard management not only saves labor resources and increases the income of growers, but also has great significance in improving resource utilization.

Fig. 1

Natural yearly cycle of fruit crops and associated management activities (outer ring) during the seasons of a year (yellow ring) (Color figure online)

Broadly, remote sensing (RS) describes a group of techniques that from a distance collect sensor data on the target of interest. This involves sensing, recording, processing, analyzing, and applying reflected or emitted energy by employing various sensing devices, such as imaging sensors, radar, and sonar and satellite sensor arrays (Usha & Singh, 2013). Recently, a considerable amount of research has demonstrated the applicability of RS in orchard management for processing the data derived from various platforms (Barbagallo et al. 2009; Garcia-Ruiz et al. 2013; Salgadoe et al. 2018). The methods, which vary according to monitoring scale, are manual observation (MO), handheld detection (HD), sensor networks (SN), ground vehicle (GV), unmanned aerial vehicles (UAVs), aerial sensing (AS), and spectral satellite sensing (SSS). These methods can be adopted for data collection relating to specific orchard management activities (e.g. thinning assessment, spraying, disease detection and yield estimation and prediction) throughout the whole fruit-trees growing cycle. Each monitoring method has its own pros and cons, depending on the application scenarios (Table 1) (Shakoor et al. 2017; Tabb & Medeiros, 2018). Although tedious, visual assessment of a limited number of trees, is the basis of a relatively accurate management strategy that depends on manual efforts to achieve the assessments (Sarron et al. 2018). Handheld detection has a similar detection range, it can provide a more accurate evaluation, and requires less expertise on fruit-tree growing on the part of the operator (Aggelopoulou et al. 2010). Compared with these two monitoring methods, ground vehicle platforms are more efficient, and extensive research on these has been conducted in the domain of orchard management (Colaco et al. 2018; Escolà et al. 2016; Wang et al. 2018).

Table 1 Relevant monitoring methods for fruit orchard management and their attributes

All three methods mentioned above are employed at ground level, and their major advantage is the quality of datasets collected—i.e., due to the close sensing distance, high-resolution data are easily accessed. However, it has now been well established in a variety of studies that remote sensing technology is of great interest within orchard management, especially when it comes to the monitoring scale and data acquisition efficiency (Salamí et al. 2019; Sola-Guirado et al. 2017; Tu et al. 2018). Conventional remote sensing technology, aerial sensing, and spectral satellite sensing have been applied in orchard management, with promising performance. However, all are limited by weather conditions and monitoring costs (Calderon et al. 2013; Panda et al. 2010). In addition, spectral satellite sensing is also limited by the lack of imagery with optimum spatial and spectral resolutions, and an unfavorable re-visit time (Berni et al. 2009b).

Thanks to recent advances in sensor miniaturization, UAVs have become increasingly available to meet the need for quick and real-time monitoring turnaround times for orchard management at usable spatial, spectral, and temporal resolutions (Berni et al. 2009a; Caruso et al. 2019; Torres-Sanchez, Lopez-Granados, et al., 2018; Valente et al. 2019). Thermal sensors, multispectral sensors, and light detection and ranging (LiDAR) systems are data acquisition resources for the direct monitoring of fruit trees. However, there are various factors to consider when choosing a UAV to carry these sensors and achieve the required time frequency and spatial resolution for orchard management. Ground-based platforms are difficult to transport from one location to another, and it may not be easy to generate field maps in real time. These limitations can be overcome by using a suitable UAV to obtain data with the required time frequency and spatial resolution (Campos et al. 2019; Matese et al. 2019). Compared with satellite-based remote sensing, using UAVs for monitoring is less dependent on weather conditions. For the monitoring of regions covered by significant cloud, UAV yielded a data-acquisition probability of 45–70%, while the probability based on satellite was around 20% (van der Wal et al. 2013). UAVs have been found to have many other advantages: they can be deployed in high-risk situations, data acquisition is fast, and images are geo-referenced (Handique et al. 2017; Saldana Ochoa & Guo, 2019). The limitations of UAVs are the battery life and operational speed. Battery life enables UAVs to operate in relatively small or medium-sized orchards. Their operational speed is lower than that of manned airborne platforms (Garcia-Ruiz et al. 2013). In order to collect data of high quality, a good balance between flying speed and data quality should be maintained during data collection.

Given the increasing need for site-specific and precise management information in orchards, the current status of UAV-based monitoring activities and research, and the potential opportunities for new applications and technologies, this paper aims to: (1) provide a detailed overview of the state of art, including UAV types, sensor types, and the analysis methodologies that have been applied, (2) identify the research gaps for fruit orchard management applications by evaluating and comparing relevant research, and (3) examine the potential technologies of UAVs for achieving precision orchard management. The hypothesis of this study is that the use of UAVs in orchard management is in its early stage and not widely investigated yet. Recent publications can provide insights in common methods and future points for development. The study is divided into three main sections. The section ‘Selection and analysis of the reviewed papers’ describes the review and analysis approach which was adopted. Next, the results section provides a detailed description of the selected literature sources using the application of UAV based monitoring approaches for orchard management as starting point, and comparing aspects related to UAV systems, camera types, analysis approaches and management indicators among literature sources. The last section, summarizes latest developments, discusses main research gaps and provides recommendations for further investigation on UAV-based acquisition approaches and orchard management oriented analysis methods.

Selection and analysis of the reviewed papers

For this review, 84 related papers from the Web of Science (WoS) database either published or available online before 1st January 2020 were found and evaluated using keyword combinations (Fig. 2). The reproducible review approach in this study is comparable to previous approaches (Leroux & Tisseyre, 2019; Pathak et al. 2019) but does not follow the strict criteria (e.g., meta-analysis) of other systematic review methods (Methley et al. 2014). Although for a selection of papers a quantitative comparison was made in this study, a complete meta-analysis could not be made because the adopted experimental designs in the studies differed too much to allow pooling of results. In order to make a sharp analysis and to avoid non reviewed references (or inaccurate information), the analysis of literature only considered scientific studies that received external review as are covered within the WoS database. This motivation is in light that the WoS core collection has been used officially by organizations as a quality standard for longer time and provides weekly updates (Falagas et al. 2008). The database of Web of Science Core Collection, which is composed of four citation indexes (Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Arts & Humanities Citation Index (A&HCI), and Emerging Sources Citation Index (ESCI)), was selected. The internal WoS search function was adopted to cover the following searching fields: the title, abstract, author keywords and Keywords Plus of each record. Timespan, language, and searching model were set for 1945–2019, English and basic search, respectively. The search string consists of two components, “monitoring platforms” and “monitored objects”, which were combined with AND:

• (1) Monitoring platforms (UAV OR unmanned aerial vehicle OR UAS OR unmanned aerial system OR remotely piloted aircraft systems OR drone)

AND

• (2) Monitored objects (orchard* OR fruit trees OR fruit crops OR pome fruits OR stone fruits OR apple OR citrus OR orange OR mandarin OR lemon OR olive OR pear OR almond OR peach OR banana OR mango OR apricot OR chestnut OR nectarine OR lychee OR coconut OR persimmon OR kiwi fruit OR avocado OR apricot)

  As noted in component (2), specific fruit species were explicitly included to ensure that the survey would not miss any related research. This search function totally generated 155 articles, which were further examined. The survey focused on the orchards planted with fruit trees such as apple, citrus, or olive. As orchards can be broadly defined as the cultivation for commercial food production of fruit crops growing as shrubs of a size between herbaceous plants and trees, or as trees, the search also yielded a considerable amount of literature on UAV-based investigation in vineyard crops (de Castro et al. 2018; Di Gennaro et al. 2019; Jimenez-Brenes et al. 2019; Matese et al. 2019; Pichon et al. 2019). But grapes fall into the category of climbing shrubs, which differ significantly biologically and geometrically from the fruit trees which form the focus of this study, so they were discarded. Similarly, articles focus on watermelon, blueberry, tomatoes, macadamia and coffee were also discarded. After removing other irrelevant ones, 80 articles remained. Among the 80 articles, one article is a meeting abstract and was discard as well (Garza et al. 2018), because not all information can be extracted. Besides these 79 articles, 4 relevant articles cited by the ones among this 79 articles were found and added to the survey (Berni et al. 2009b; Ishida et al. 2018; Torres-Sanchez et al. 2015; Zarco-Tejada et al. 2013). One article was actually published in 2019 but its publication year in WoS system was 2020 (Martinez-Guanter et al. 2019). It was also included.

Fig. 2

Overview of the criteria and categories investigated for the papers (84 in total) selected from the Web of Science for the survey

For every paper, specifications of the study (e.g., sensors applied, study regions) were derived and compared (Fig. 2). First, the selected papers were classified into five categories on the basis of their main focus: resource efficiency, geometric traits, productivity, disease, and other applications. In each category, relevant papers were analyzed in relation to the following aspects: research objectives, UAV type(s), sensors uploaded, analysis methods applied, and results in relation to orchard management. To identify factors that affect the research performance, comparisons among fruit species were also made. As a result, this paper is concluded by summarizing the state of art, potential of new technology, research gaps, and opportunities for UAVs in orchard management.

Results

General status

The trend in UAV-based orchard management publications and citations of these articles is illustrated in Fig. 3. Publications on UAV-based orchard management show a steady increase over the period from 2009 to 2019, in spite of a small drop around 2015. As can be seen from the citation statistics (3.b), UAV-based orchard management has attracted significant interest, hence the need for comprehensive analysis and summary at this stage. The publications originate from and report fieldwork in a large variety of countries throughout the world (Fig. 4), and they deal with different fruit species, mainly depending on climatic and geographic conditions.

Fig. 3

Source: Web of Science)

a Number of publications per year for the period 2009–2019 and b Annual cumulative citation frequency. The search was conducted on January 01, 2020 (

Fig. 4

Distribution of fruit species involved in the 84 research conducted in different countries. Citrus includes the fruit species of orange, mandarin and lemon

The articles reviewed were classified into five classes on the basis of the orchard management application of the UAV and the objective of the application (Fig. 5):

• Resource efficiency: To optimize the efficiency of the inputs: e.g., water, site-specific status, or even status of individual trees is monitored and subsequently used to devise an irrigation strategy.

• Fruit-crop geometric traits: To monitor the dynamics of fruit tree growth and potential yield, geometric traits such as tree height, canopy volume, and area are evaluated. The measured data can further be used for pruning effect assessment or pruning planning.

• Fruit-crop productivity: In order to maximize economic benefits, the spatial yield estimation and prediction of final yield at final harvest are investigated.

• Fruit-crop disease: In regular monitoring of the health status of fruit crops, disease classification and assessment should be timely, in order to provide basic protection.

• Other applications: A few studies focused on UAV-based pesticide spraying, others reported research on agricultural resource management and food quality tracking.

Fig. 5

Distribution of the five application categories in the selected 84 papers. RE resource efficiency, GT geometric traits, P productivity, D disease, OA other applications

Aspects for the entire fruit-crop growing cycle (Fig. 1) were covered, but there were significant differences in the focus: the three main categories—resource efficiency, geometric traits and productivity—account for 67% of the research conducted in orchard management. Below, all the five application fields will be used as a starting point for further comparison and synthesis of the selected papers.

UAV platforms

An aircraft flying in a set direction and at a set speed and controlled remotely is referred to as an unmanned aerial vehicle (UAV) or drone. In recent years, the availability of UAVs has rapidly increased, and there are now many types, from multi-rotor to fixed-wing. According to the literature reviewed in this paper, three types of UAVs are currently used for orchard management: fixed-wing, rotary-wing, and multi-rotors (Table 2).

Table 2 Typical types of UAVs used for orchard management in five of the 84 papers

The specific UAV type chosen for orchard management depends on the requirements and limitations of the application. The most widely used UAV types in orchards are multi-rotors. A multi-rotor is a helicopter that can fly at different altitudes; it can be driven by four to eight rotors. This platform has several advantages. It can hover over a given location, use global positioning system (GPS)—based navigation, fly horizontally and vertically, and only requires a small take-off and landing space. The ease of control and high maneuverability are also the benefit of it. In addition, a flight altitude of 70 m can enable the optical sensor to achieve a 3D resolution of centimeter level already (Caruso et al. 2019). Yet, the main limitations of rotor craft compared to fixed-wing craft are the lower speed and shorter flight time.

Compared to multi-rotor UAVs, fixed-wing UAVs have longer flight times and faster travel speeds. A major advantage of fixed wing UAVs is the capability of high payload though the cost of this UAV is relatively higher than multi-rotor UAVs. However, they are not able to hover. The speed of this type of UAV can cause image blurring, which can be solved by using an imaging sensor with improved features, including fast shutter speeds. Moreover, recent developments in the field of UAV have led to an interest in the adoption of UAVs with vertical take off and landing (VTOL) system in orchard management (Mesas-Carrascosa et al. 2018; Torres-Sanchez, Lopez-Granados, et al., 2018). VTOL UAVs are easy to maneuver and have the freedom from bad weather and even site condition restrictions. In addition to the UAVs mentioned above, customized UAVs have also emerged in order to meet particular requirements in some case studies (Stefas et al. 2019). These UAVs are regarded as imaging and remote sensing platforms in orchard management. Still, UAVs also play an active role, such as the UAVs applied for spraying, which provide a new solution for the safety of conventionally manual pesticide spraying (Gao et al. 2019).

Sensors

UAVs have been equipped with various sensors and have been able to collect data on the color, spectral, temperature, and geometric traits of fruits or fruit trees. So far, the main sensors used for UAV-based orchard management (UAV-OM) are RGB and multispectral. Hyperspectral and thermal sensors are used in a relatively small range of applications in UAV-OM (Fig. 6). In the literature reviewed, research using LiDAR only was described in one paper, which focused on the geometric traits measurement of apple trees (Hadas et al. 2019). The application of LiDAR is limited by cost and power consumption, especially in the case of lightweight UAVs (Stefas et al. 2019). Depending on the application scenario, the five main sensor types play different roles in different orchard management activities (Fig. 6).

Fig. 6

Sensors for UAV-based orchard management as mentioned in the 84 papers selected for this survey. The areas of the brightly colored circles in the figure represent the proportion used in similar applications; specific number of relevant articles are also marked; rounded rectangles represent zero. The abbreviations are the initials of the application category: “R” stands for Resource efficiency in orchards (dark blue), “G” for Geometric and biophysical traits (light blue), “P” for the applications in Productivity (yellow), “D” for Disease detection (orange), and “O” for Other applications (green). Because LiDAR was employed only once in the literature reviewed, it is not included in the figure (Color figure online)

UAVs equipped with RGB imaging sensors have the widest range of applications in visible characterization, such as fruit-tree geometric trait assessment using grayscale or color images (Ok & Ozdarici-Ok, 2018a; Xue et al. 2019). RGB sensors, also referred to as “point-and-shoot cameras” and “consumer-grade cameras”, are the most commonly used sensors in orchard research. Their advantages are low cost and simple operation. The high-resolution RGB sensor in particular is capable of providing detailed information for a wide range of orchard management applications (Gao et al. 2019). For example, the sensor provides sufficient detail for analyzing the physiological characteristics of peach trees when ground resolution of RGB images fell into the range of 0 ~ 1 cm/pixel. The correlation between orthogonal crown widths derived from UAV imagery and measured in the field yielded an R² value of 0.91 (Mu et al. 2018). Normally, users can obtain high-quality images by adjusting the exposure parameters of the sensor according to weather conditions. In the domain of UAV-OM, orthophotos or digital surface models (DSMs) / digital elevation models (DEMs) indirectly obtained from RGB are of most interest to researchers (Jimenez-Brenes et al. 2017; Ok & Ozdarici-Ok, 2018b). First, in order to obtain high-quality orthophotos, to minimize the influence of camera distortion RGB images need to undergo deformation processing based on the specific model of sensor used. Then, algorithms are used for image matching and optimization processing. Finally, automatic processing software such as Agisoft Photoscan software (Uribeetxebarria et al. 2019) is used to stitch the acquired images into a complete ortho-mosaic.

Depending on the spectral resolution, multispectral and hyperspectral sensors have been used in a variety of UAV-OM applications, from water status assessment to disease detection (Calderon et al. 2013; Ishida et al. 2018; Romero-Trigueros et al. 2017). The primary imaging principle of these sensors is based on differences in spectral absorption and reflection characteristics of different objects or different parts of the same object. The image processing for the spectral sensor mainly includes radiometric and geometric pre-processing with subsequent statistical analysis. Compared with RGB sensors, datasets generated from hyperspectral and multispectral sensors have increased processing effort, and standardized processing chains are still being developed. Processing the hyperspectral data is a complex task, and the hybrid spectral decomposition model has certain limitations in practical research (Guillen-Climent et al. 2012). In terms of the cost, multispectral sensors are relatively affordable for the growers in orchard but, it acquires spectral information in lower bands than do hyperspectral sensors—and hyperspectral sensors have an advantage in terms of spectral detail: they can record continuous spectra, covering a large part of the optical range. This feature enables hyperspectral sensors to perceive the spectral characteristics and spectral differences of fruit trees, making them ideal for applications where specific traits need to be derived (Abdulridha et al. 2019).

With their combination of an infrared detector and an optical lens, thermal sensors can receive infrared radiation energy in the emissive part of the spectrum. The temperature differences that can be derived from these thermal radiance observations are especially useful for the assessment of water status (Gomez-Candon et al. 2016; Park et al. 2017). The primary sensor for assessments of water status is the thermal sensor, as there is a close relationship between transpiration rate and canopy temperature due to photosynthesis (Zarco-Tejada et al. 2012). In the measurement of crop canopy temperature, the selection of the region of interest varies and influences the estimation accuracy (Stagakis et al. 2012). UAVs equipped with thermal sensors are capable of recording fruit-tree canopy temperatures which reveal the temperature differences in canopies under different conditions. However, during this process, solar radiation, air temperature, and wind speed around the canopy also have an impact on data collection, making the final assessment results uncertain. In addition, in order to simulate the overall energy balance of the working environment, it is also important to create models which are relevant, such as the tree canopy conductance model proposed in previous research, which incorporates simulations of net radiation and aerodynamic resistance (Berni et al. 2009a). Meanwhile, sensor correction and processing of mixed pixels are still a problem that cannot be ignored (Gomez-Candon et al. 2016). Fortunately, data fusion, such as the fusion of thermal and RGB images, shows potential for providing a solution.

Advances in UAV-based remote sensing in orchard management

To enable the comparison among studies with the same or similar objective, a UAV-based orchard management framework was proposed (Fig. 7). The selected literature fell into the proposed five categories according to their research objectives in terms of the management activities introduced in Fig. 1. Various data sources were acquired for different management scenario showing notable difference. LiDAR sensor was applied for geometric traits estimation only. Next, decision indicators were extracted or calculated from the collected datasets utilizing advanced methodologies. Each “orchard management” sub-category contains important aspects of the management activities, such as thermal drift correction which aims to improve the accuracy of assessment of water stress. In each application scenario, different studies share the same main focus.

Fig. 7

Data-decision framework in orchard management using UAV technology, based on the selected 84 papers

Fruit-tree geometric traits

Geometric characteristics such as the shape and size of trees have emerged as important indices for a wide range of activities for managing the growth process of fruit trees. In breeding trials, information on olive tree crown dimensions provides a benchmark for developing suitable cultivars in a given training system, e.g., open vase configuration or hedgerow (Ben Sadok et al. 2012; De la Rosa et al. 2007). Here, a training system means the management model for growing the fruit trees to a desired size and form, which is accomplished by pruning. Furthermore, traits like canopy area and crown volume are decisive when assessing pruning impact. By mapping these traits, optimal pruning type and intensity can be selected and applied (Castillo-Ruiz et al. 2015; Miranda-Fuentes et al. 2015). In general, structural properties monitoring is capable of elucidating tree crop growth status. Further, these geometric traits are useful for developing site-specific treatments involving water, and for ameliorating the management problems caused by soil heterogeneity. Acquiring conventional measurements manually requires intensive effort and is associated with much uncertainty due to the irregularity of tree crowns. First, the primary dimensions (e.g., the tree height) are measured and empirical models or equations that represent the trees as regular polygons are applied for the characterization of the trees (e.g., the canopy area or crown volume) (West, 2009). In-situ measurements of large orchard plots are more inefficient and costly. The estimation from terrestrial platforms show promising performance; they include active RS technology and LiDAR laser scanners, which have been found to be able to achieve an R² value of 0.97 for the tree height estimation (Moorthy et al. 2011). Though UAVs equipped with LiDAR have not been properly explored for geometric measurement in the domain of orchard management, UAVs installed with other sensors, e.g., RGB and multispectral, have been shown to be reliable alternatives for fruit-tree geometric measurements (Anifantis et al. 2019; Hadas et al. 2019).

Recent studies have described UAV-based quantification of geometric features in fruit trees, focusing on automated 3D reconstruction technique. 3D reconstruction of trees is the one of challenges in the domain of remote sensing, whereas 2D delineation quality affects the estimation based on the 3D digital models (Ok & Ozdarici-Ok, 2018a). Without the use of height thresholds, a delineation method for citrus-tree canopies that employs orientation-based radial symmetry transform (OBRS) resulted in an overall F1-score of 91.2% (Ok & Ozdarici-Ok, 2018a). OBRS transform locates the circular objects directly and improves the accuracy of the subsequent extraction of regions of interest. Circular Hough transform algorithms are also suitable for the extraction of fruit trees. Research has indicated that an approach combining this algorithm with sequential thresholding and canny edge detection resulted a delineation accuracy exceeding 80% (Koc-San et al. 2018). Neural network based methods are also capable of classifying fruit trees (Ampatzidis et al. 2019). For instance, (Kestur et al. 2018) compared the K-means method with an extreme learning machine (ELM) approach that uses a single hidden layer feed forward neural network (SLFN) for the classification of different fruit-tree crowns (banana, mango, and coconut). In this case study, ELM performed better than the unsupervised K-means method: the classification accuracies were 96.0% and 85.5%, respectively.

In general, the combination of geographic object-based image analysis (GEOBIA) and DSMs derived from the structure from motion (SfM) method is a well-established approach for fruit-tree classification and geometric index calculation in which the indices consist of tree height, and crown width and perimeter (Jimenez-Brenes et al. 2017; Johansen et al. 2018; Ok & Ozdarici-Ok, 2018b; Torres-Sanchez et al. 2015). In a case study of height estimation, a GEOBIA method treating points between 1 and 1.5 m as the crown center was first applied in photogrammetric point cloud analysis, yielding an R² value of 0.94 (Torres-Sanchez, de Castro, et al., 2018). This indicates that dense 3D point clouds are sufficiently representative to be used for geometric measurements. With the use of similar method, the differences in the quality of tree crown 3D reconstruction derived from two training system, intensive and hedgerow system, was observed (de Castro et al. 2019). Furthermore, when a random forest classifier was trained on the basis of GEOBIA, the estimation of crown height and plant projective cover (PPC) yielded a R² value of 0.65 and 0.62, respectively (Tu et al. 2019). In the case of peach trees, (Mu et al. 2018) reported that a combination of adaptive threshold and watershed segmentation methods was suitable for measurements of crown width and crown projection area (CPA, m²) (Eq. 1). Based on this, the spatial and temporal data on growth rate based on CPA was observed. An adaptive threshold is also appropriate for filtering non-fruit trees, particularly the threshold related to tree height (Xue et al. 2019).

$$ {\text{CPA}} = { }0.65 \times {\text{W}}_{1} \times {\text{W}}_{2} $$

(1)

where the equation is based on local experience, and $W_{1}$, $W_{2}$ represent the crown widths parallel and perpendicular to the tree rows.

Additionally, the UAV data collection design has a significant impact on the estimation results for different application scenarios. (Torres-Sanchez, Lopez-Granados, et al., 2018) indicated that the best configuration for olive-tree volume estimation is the combination of 95% forward overlap and 60% side overlap while the flight altitude, the ground sampling distance (GSD) and image footprint was 100 m, 0.038 m/pixel and 124*94 m, respectively, which could achieve an estimation accuracy of 95% and save 85% computing time compared to applying maximum overlap (97%). The effects of flight height on the tree height estimation was also validated. It showed that the R2 value changed from 0.79 to 0.86 when the GSD decreased from 16 to 3 cm (Marques et al. 2019). GSD value affects the point cloud density directly. Research indicated that the row detection accuracy could reach 100% when the GSD ranged from 2.13 to 6.69 cm/px (Sun et al. 2019). A approach in order to enable automatic estimation of geometric parameters has been to develop valuable processing tools in QGIS software. One new plugin was capable of achieving automatic detection of trees by integrating several external algorithms and had a classification accuracy of 92.84% (Duarte et al. 2018). Different spatial resolutions produced DSMs of inconsistent quality. For one study, a stable relationship between spatial resolution and DSMs quality was found when resolution fell in the range of 5 to 30 cm/pixel (Zarco-Tejada et al. 2014). To date, limited attention has been paid to the feasibility of upscaling methods, and several methods have only been evaluated for a given training situation (Torres-Sanchez, de Castro, et al., 2018). The methods should be tested in other circumstances, e.g., related fruit species, other data collection strategies. Focusing on the isolation of error sources in the full process of estimating the geometric traits of fruit trees can likely identify the critical steps in workflow. For instance, methods based solely on morphological traits such as shape are ineffective if the background contains objects of no interest that have the same morphological traits.

Fruit-tree productivity traits

Productivity estimation is frequently prescribed for orchard management practices as it provides key information for growers and other stakeholders in market supply and exports. Detailed spatial explicit information in particular is key for growers to facilitate efficient utilization of resources and to optimize and streamline their harvest process (Suo et al. 2019; Woodward & Clearwater, 2012). In addition, estimation of essential elements such as the nitrogen status and chlorophyll content and of the fraction of intercepted photosynthetically active radiation (fIPAR) during the growing season also benefits the estimation of potential final yield and fruit-crop performance (Guillen-Climent et al. 2012; Perry et al. 2018; Vanbrabant et al. 2019). Traditional in-situ estimation of productivity variables is time-demanding and uncertain. It entails visual inspection of number, color, shape, size, and other information on fruits or fruit trees according to the grower’s own experience (Srivastava & Sadistap, 2017). This results in limited samples and repetitions, which are insufficient to account for the spatial and temporal variability within and between orchards (Aggelopoulou et al. 2009; Perry et al. 2009). Most studies, however, have been conducted under special conditions (e.g., using light-shielding cover to collect data), or at ground-scale level, which also shows the limitations for operational management (Nguyen et al. 2016). A recent study has examined yield estimation based on satellite imagery, but the method was shown to have low feasibility for different orchards (Rahman et al. 2018). Despite the timeliness and low monitoring cost of satellite-based methods, they do not satisfy all the requirements of yield-related management in orchards.

It is now well established from a variety of studies that UAV-based imaging has potential to support data-driven yield estimation algorithms. In general, counting flowers/fruits on trees is the direct and precise solution using UAV imagery technology (Horton et al. 2017). Yet, the occlusion of flowers/fruits by branches, leaves and/or other fruits, variable outdoor lighting conditions, and color similarity seriously affect the direct estimation. Fruit trees need to be accurately thinned during flowering stage in order to gain better yield and fruits with good quality. Thus the basis is the precise estimation of the flowering status in orchards. In this context, an enhanced bloom index (EBI) (Eq. 2) was proposed for quantifying floral phenology in an almond orchard. The EBI is capable to enhance flower signals and reduce the noise produced by soil and green vegetation. Robust information on floral phenology was provided by the EBI, showing an agreement with the bloom coverage (with a R² of 0.72) (Chen, Jin, et al., 2019). Flower numbers is difficult to accurately count with aerial images due to the high density and occlusion of flowers. To deal with this, flowering density calculated from 3D point clouds was proved to monitor the flowering dynamics at field scale using sfm and OBIA approaches (Lopez-Granados et al. 2019). To further improve the feasibility of UAV in yield mapping, a vision-based UAV system was suggested (Stefas et al. 2019). Its capability of avoiding obstacle in high density orchards enable increasing spatial resolution navigating autonomously among apple rows and trees.

$$ EBI = \frac{Brightness}{{Greenness \cdot Soil \cdot Signature}} = \frac{R + G + B}{{\frac{G}{B} \cdot \left( { R - B + \varepsilon } \right)}} $$

(2)

where the $\varepsilon$ is an adjusting constant to make the denominator non-negative, and R, G, B are the reflectance of red, green and blue bands, respectively.

In addition to direct measurements, alternative parameters (such as geometric traits and vegetation indices) for indirect productivity estimation or prediction are of interest. Canopy projected area (CPA) and canopy perimeter have been proved to be correlated with the fruit load of peach trees (Uribeetxebarria et al. 2019). CPA yielded a higher correlation, with an R value of 0.85. Olive yield can be forecast from individual crown area estimated from UAV images (Sola-Guirado et al. 2017), and a high agreement between the crown volume and yield was also observed in some almond varieties (Lopez-Granados et al. 2019). In the case of banana productivity, normalized difference vegetation index (NDVI) was positively correlated with several metrics regarding the yield and fruit quality (e.g., bunch weight, length of the longest finger), and negatively correlated with fruit loss (Machovina et al. 2016). Further, an attempt to demonstrate the influence of soil heterogeneity on fruit productivity found no relationship between NDVI and physical soil quality. In addition, predictive models for mango yield based on geometric parameters have provided an R² value greater than 0.77 using GEOBIA, without counting numbers (Sarron et al. 2018). Meanwhile, research exploring whether the methods proposed have universal application is interesting. One source of weakness in estimation using structure indices is that vegetation dynamics related to short-term physiological processes cannot be captured. To deal with this, chlorophyll content related index—TCARI/OSAVI, light use efficiency related index—PRI570, and canopy chlorophyll fluorescence can be alternatives. Research suggests that these three indices yielded values of R² in the range between 0.75–0.84 when estimating the gross primary production (GPP) (Zarco-Tejada et al. 2013). Productivity estimation related to physiological is largely based on data derived from multispectral or hyperspectral sensors. In this context, there are still operational challenges in employing fixed-wing UAVs (two hours’ preparation time: set up, camera calibration, safety check, and launch) and in application scale (Machovina et al. 2016).

In order to optimize fruit productivity and harvest activities, it has been advised to focus on harvesting time optimization, fruit tree vitality and health status monitoring (Vanbrabant et al. 2019). Fruit ripeness is the key element to determine the harvest time; it affects fruit quality directly during transport to markets. Preliminary efforts on ethylene detection via ethylene-sensitive sensors attached to UAVs have shown the effects of flying height and sensing wind speed when assessing apple maturity (Valente et al. 2019). This demonstration provides a novel method for harvest time optimization, though the modeling and simulation results indicated a short detection margin for the ethylene. In one study in the domain of radiation interception estimation, vegetation indices mostly related to tree crop structure were regarded as a proxy for fraction of absorbed photosynthetically active radiation (fAPAR) (Zarco-Tejada et al. 2013). On the other hand, fIPAR at crop canopy scale also proved mappable via airborne imagery, especially in peach and citrus orchards (Guillen-Climent et al. 2012). Similar fIPAR estimation results were found when methods employing the combination of 3D radiative transfer model and scaling-up were compared with a model inversion method: RMSE values were 0.09 and 0.10, respectively. It was shown that row orientation affected the relationship between NDVI and fIPAR. Further, fIPAR quantification was also validated via Mahalanobis distance distance supervised classification method, which resulted in a RMSE of 0.06 (Guillen-Climent et al. 2014). In order to achieve higher productivity, regular nutritional status monitoring is the main requirement for fruit species like citrus (Osco, Marques Ramos, et al., 2019). Evidence was found that nitrogen contents prediction accuracy for citrus depends on the sub-tree areas where the spectral data extracted from. Nitrogen prediction accuracy based on the spectral data from the whole canopy and the young leaves was found lower than that from the mature leaves, while the effects of sub-tree areas on the prediction accuracy of soluble sugar and starch in the leaves were not clear (Liu et al. 2016). Instead of using conventional indices for nitrogen status assessment, (Perry et al. 2018) applied a new index, the modified canopy chlorophyll content index (M3CI) (Eq. 3), for the assessment at canopy level, resulting in an R² value of 0.67. Machine learning algorithms currently performed well in the monitoring of fruit tree conditions such as chlorophyll content (Vanbrabant et al. 2019), nitrogen content. Compared with SVM, ANN and decision tree (DT) machine learning algorithms, random forests (RF) was the optimal one for the prediction in canopy nitrogen content of orange trees. R2 value reached 0.9 while the mean squared error (MSE) was 0.307 g/kg (Osco, Ramos, et al., 2019).

$$ {\text{M}}3{\text{CI}} = { }\frac{{{\text{R}}_{{{\text{NIR}}}} + {\text{R}}_{{{\text{Red}}}} - {\text{R}}_{{{\text{RE}}}} }}{{{\text{R}}_{{{\text{NIR}}}} - {\text{R}}_{{{\text{Red}}}} + {\text{R}}_{{{\text{RE}}}} }} $$

(3)

where the $R_{NIR}$ was measured in the band of 810-nm, the measured reflectance of $R_{Red}$ and $R_{RE}$ were in the 660-nm and 710-nm band, respectively.

The aforementioned papers still reveal limitations in the data-processing time (Sarron et al. 2018), the robustness of methods proposed, and experimental design in terms of sample size (Perry et al. 2018). Finally, nearly all of the research was conducted at a certain period of time and few comparative studies report results of estimations at different fruit-growing stages throughout the entire growth cycle.

Resource efficiency in orchards

The water used for irrigation in agriculture accounts for 85% of the total water managed at a global scale. Because of the water shortage worldwide, precise water management in orchards is a crucial practice, particularly in semi-arid areas where water inputs require higher economic investment. Climate change is also becoming a problem for the fruit industry, as some geographic areas are experiencing long periods of drought (Gomez-Candon et al. 2016). An efficient irrigation strategy is key to minimizing this negative impact on orchard managers’ profit. Sufficient water input is closely related to fruit production and quality. Even some drought-tolerant species like olive (Olea europaea L.) can benefit from irrigation, i.e., the promotion of growth, yield and fruit quality (olive oil), especially when grown in high-density training systems (Caruso et al. 2019; Egea et al. 2017). Within the definition of deficit irrigation, water inputs should meet the realistic requirement of the trees. Site-specific water management is thus of great importance to track the variability of water needs in orchards. More specifically, it solves the variability problem caused by soil heterogeneity and canopy-cover differences (Couvreur et al. 2016). The use of remote sensing technology for fast assessment of the water status in orchards aims to improve productivity and water use efficiency in irrigation. For instance, the normalized difference red-edge (NDRE), derived from UAV imagery is capable for monitoring the irrigation inhomogeneities and may further identify the growth inhomogeneities (Jorge et al. 2019).

When water supply from the soil cannot meet the demand for transpiration of plants, water deficits occur. These short-term deficits may hamper the growth of fruit trees and will affect the productivity. As most fruit trees are sensitive to water deficits, information related to optimizing irrigation is critical. Water stress monitoring benefits the application of deficit irrigation, allowing water resources to be saved and fruit yield and quality to be maintained (Girona, 2002; Zarco-Tejada et al. 2012).

In the late 1970s, canopy temperature was identified as a proxy for water status monitoring, with the concept of crop water stress index (CWSI) (Idso et al. 1978; Jackson et al. 1981). The CWSI normalizes the difference between air (T_a) and canopy temperature (T_c), and lower (LL) and upper limit (UL) (transpiration of a leaf at potential rate and no transpiration, respectively), demonstrating the evaporative demand (Eq. 4).

$$ {\text{CWSI }} = \frac{{\left( {{\text{T}}_{{\text{c}}} - {\text{ T}}_{{\text{a}}} } \right){ } - { }\left( {{\text{T}}_{{\text{c}}} - {\text{ T}}_{{\text{a}}} } \right)_{{{\text{LL}}}} }}{{\left( {{\text{T}}_{{\text{c}}} - {\text{ T}}_{{\text{a}}} } \right)_{{\text{UL }}} - { }\left( {{\text{T}}_{{\text{c}}} - {\text{ T}}_{{\text{a}}} } \right)_{{{\text{LL}}}} }} $$

(4)

UAV imagery has shown to be potentially more efficient for canopy temperature assessment than traditional field measurement (Gonzalez-Dugo et al. 2014). When it comes to the exploration of canopy temperature, stomatal aperture is key to understanding the fluctuations, i.e., stomatal closure can lead to a decrease of evaporative cooling and rise in leaf temperature. The impacts of environmental conditions on stomatal response vary for different fruit species, which should be borne in mind, especially when the temperature of the fruit-tree canopy is taken as an indicator (Ballester et al. 2013). CWSI applications are mainly restricted by two aspects: necessary spatial resolution and the site-dependent equation of non-water-stressed baseline (NWSB). Without the use of reference surfaces, a high-resolution CWSI map was achieved by combining energy balance equations based on physical models with thermal imagery (Berni et al. 2009a). This reveals the capability of CWSI for quantifying the spatial variability too. However, if the targeted orchard contains different species or the same fruit species with different training systems, the thermal response is affected and a single set of reference values may lead to errors. In a study investigating this case (Park et al. 2017), an adaptive CWSI yielded an agreement with stem water potential ($\psi_{s}$) and stomatal conductance (${\text{g}}_{{\text{s}}}$) with determination coefficients (R²) of 0.72 and 0.82, respectively, employing temperature thresholds. This was in contrast to the conventional CWSI, which yielded R² values of 0.27 and 0.34, respectively. The orchard had been divided into four sub-areas according to the fruit species and training systems and the adaptive thresholds of the lower and upper reference were estimated for the adaptive CWSI calculation. The CWSI algorithm applied is shown in Eq. (5). Here, the canopy temperature derived from aerial imagery was applied instead of the difference between canopy and air temperature (Jones, 2013).

$$ {\text{CWSI}} = \frac{{{\text{T}}_{{\text{c}}} { } - { }\left( {{\text{T}}_{{\text{c}}} - {\text{ T}}_{{\text{a}}} } \right)_{{{\text{LL}}}} }}{{\left( {{\text{T}}_{{\text{c}}} - {\text{ T}}_{{\text{a}}} } \right)_{{\text{UL }}} - { }\left( {{\text{T}}_{{\text{c}}} - {\text{ T}}_{{\text{a}}} } \right)_{{{\text{LL}}}} }} $$

(5)

where $T_{c}$ is the aerial canopy temperature measured, $T_{a}$ is the air temperature, LL represents the temperature of a leaf at full transpiration and UL is non-transpiring temperature.

It is widely accepted that $\psi_{s}$ is a reliable parameter which is closely related to plant response to water stress.$\psi_{s}$ not only reveals the water status in the plant–soil–atmosphere continuum but integrates the effects from soil moisture and evapotranspiration. However, obtaining $\psi_{s}$ values with a field measurement method is a labor-intensive and inefficient process (Zhao et al. 2017); alternative indices derived from aerial imagery are therefore needed. Thus, in addition to CWSI, research on comparing vegetation indices derived from the UAVs with ground-collected data, such as $\psi_{s}$ and ${\text{g}}_{{\text{s}}}$, is currently of interest (Gonzalez-Dugo et al. 2013; Stagakis et al. 2012; Zarco-Tejada et al. 2012).

To explore suitable indicators for the assessment of water stress, a variety of reflectance indices derived from UAVs have been calculated, analyzed, and compared with water-stress-related measurement methods conducted on the ground, e.g., ${\text{g}}_{{\text{s}}}$ (Table 3). Indices integrating data from different spectral ranges such as the NDVI have shown potential for inferring water status in several types of orchards (Caruso et al. 2019). Indices such as chlorophyll and fluorescence indices (leaf-level), green ratio (GR), enhanced normalized difference vegetation index (ENDVI), normalized difference green near infrared index (NDGNI), and saturation (S) have also proved to be sensitive to water status (Bulanon et al. 2016; Zarco-Tejada et al. 2012). Intensity (I) did not provide reliable results in the case of apple orchards. Additionally, two formulations of photochemical reflectance index (PRI), PRI₅₇₀ and PRI₅₁₅, were correlated with water stress and showed promise for fruit quality assessment in an orange orchard (Stagakis et al. 2012). NDVI calculated at canopy level in almond trees decreased within areas with high water stress when the blue band was employed (Eq. 6), which indicates that the canopy NDVI could be a water stress indicator for some fruit crops (Zhao et al. 2017).

$$ NDVI_{B} = \frac{{{\uprho}_{{{\text{NIR}}}} - {{\uprho }}_{{\text{B}}} }}{{{\uprho}_{{{\text{NIR}}}} + {{ \uprho }}_{{\text{B}}} }} $$

(6)

Table 3 List of spectral vegetation indices used in the assessment of water stress and its associated equations and applied for UAV-acquired datasets in fruit orchards

On the other hand, as a canopy structure parameter, the leaf area index (LAI) is sensitive to water stress. Studies of LAI indirect measurement showed that the NDVI calculated from UAV images also correlated with the LAI measured on the ground (R² value ranged from 0.78 to 0.88) (Berni et al. 2009b; Caruso et al. 2019).

However, not all the pixels within the canopy area yield the water status: these are non-leaves or shaded areas. Besides the multispectral indices mentioned in Table 3, monthly canopy volume increment correlated well with daily water stress integral (WSI), with an R² of 0.99 (Caruso et al. 2019). WSI reduces the impact of the fluctuations in water status. The slope of (T_c—T_a) over time was found to be another novel indicator (Gonzalez-Dugo et al. 2013). Generally, research focused more on the indices related to canopy structural changes than on diurnal physiology changes (Romero-Trigueros et al. 2017). In the case of orchards with several fruit species, using one single index could dramatically simplify management. Because different species vary in their canopy architectures and nutrient status, diverse water status indicators are needed (Ballester et al. 2018). Suitable indices should not be determined by numerous specific conditions, such as fruit species, irrigation methods, and geographic conditions (Bulanon et al. 2016; Caruso et al. 2019). For particular applications, varying experimental comparisons are required, and a general case study per application may provide insight into the bottlenecks. For example, in order to explore the universal vegetable index for water status assessment, the performance of methods proposed for different fruit species should be tested.

Generally, two thermal sensor systems are available for temperature imaging: cooled systems, which are loaded on satellite and aerial platforms, and uncooled systems, which are used on UAV payloads with less power consumption. However, the temperature drift that occurs in uncooled systems affects the rate of error, causing offset non-uniformity of the acquired data (Gomez-Candon et al. 2016). Some drift correction strategies for thermal sensors need additional flying time, which means higher requirements for on-board batteries, though the accuracy is greater than 1 ℃. Maintaining the same correction accuracy and using the methods based on redundant information, the cubic drift model enables more efficient drift correction (Mesas-Carrascosa et al. 2018). Even during data collection prior to this operation, the final estimation of water status can be adversely affected by factors such as: the effects of solar motion when aiming at calculating canopy NDVI; the data collection interval, which may reduce the influence of fluctuation; and the flight altitude, which affects subjective interference. Fruit orchard properties (e.g., cultivars and training systems) should be taken into account in order to achieve a comprehensive analysis of the images. Conducting edge extraction prior to modeling enhanced the mapping accuracy of stem water potential (Park et al. 2017). Yet approaches on increasing the efficiency of image extraction and radiometric correction should also be considered for use in the statistical analysis or image processing (Gomez-Candon et al. 2016). In addition, it is not uncommon for pixels to be mixed together in image areas, e.g., areas contain canopy and non-interested pixels, the soil pixels, and methodologies need to be further developed to reduce the associated error. Though in most cases the fine-resolution imagery necessary for pure crown extraction or intra-canopy variability investigation is available, more quantification studies are needed for precise water management in orchards.

Detection of diseased fruit trees

Different types of diseases may occur throughout the fruit-growing season—from flowering to harvest, and even in the dormant tree period during winter. In terms of their cause, diseases in orchards fall into two categories: biotic and abiotic. Biotic diseases are caused by living pathogens, which could be bacteria, fungi, viruses, or insects. A well-known apple bacterial disease is fire blight, which leads to significant losses in fruit production by infecting the fruits and rootstock of fruit trees (Jarolmasjed et al. 2019). Apple scab and pear black necrotic leaf spot belong to fungal and viral diseases, respectively (Belfanti et al. 2004; Shim et al. 2004). Abiotic diseases are caused by the interaction of fruit trees and other factors in the planting environment, such as water pollution, overwatering, and extremes of light and nutrients. Treating abiotic diseases as abiotic disorders may predispose fruit trees to infectious diseases, and abiotic and biotic diseases can also occur in the same trees. Diseases significantly affect the fruit quality and final yield by infecting fruits, trees, and other areas, such as twigs and leaves. Many diseases show mild effects, resulting in limited to no harm at some point. But some diseases even cause tree mortality. Huanglongbing (HLB), or citrus greening, and phytophthora root rot disease can be lethal to fruit trees worldwide and attract growers’ attention because of their significant economic impacts (Salgadoe et al. 2018; Sankaran et al. 2011). The most effective management to deal with fruit trees diseases is to detect the infected trees as early as possible. In addition, specific treatment can be taken, i.e., removing diseased trees, applying dedicated pest protection measures, and planting resistant species. Some diseases are hard to diagnose from visual symptoms at the early infected stage, with the result that no effective action can be taken to deal with the disease when the serious symptoms are recognized at a late stage. Traditional methods for diagnosing fruit-tree diseases are visual observations in the field combined with laboratory analysis and have limitations relating to reliable evaluation and time–cost efficiency (Khan et al. 2018; Pan et al. 2014; Srivastava & Sadistap, 2017). UAV-based fruit-crop disease monitoring has been employed for a few types of disease, but it is still critical to investigate its applicability for monitoring severe diseases like Panama disease in banana (O'Neill et al. 2016). Additionally, disease detection based on aerial images from UAVs can provide orchard scouting over a larger area and is low-cost in terms of both time and equipment.

Disease identification is the first step for practical control. The complexity of disease diagnosis and diversity of fruit species hampers the transfer of research findings to other methods of fruit-tree disease detection. Different symptoms of diseases have been classified with different sensors, deriving valuable indicators from aerial images. The capability of UAVs equipped with hyperspectral and multispectral sensors to classify citrus trees infected by two types of biotic diseases, bacterial canker and HLB, respectively, has been described (Abdulridha et al. 2019; Garcia-Ruiz et al. 2013). Machine learning has shown promise for exploring the complex sensitivity of an indicator for a specific disease diagnosis issue. Citrus bacterial canker (symptoms are yellow halos on fruit and twigs) is a disease with serious implications (Duan et al. 2018). (Abdulridha et al. 2019) explored detection techniques for this disease under laboratory conditions and in an orchard, utilizing hyperspectral imaging and machine learning. In total, 31 vegetation indices were evaluated for the disease detection; also studied was the classification of disease development stages—asymptomatic (infected but with no symptoms), early (tiny lesions), and late symptoms (brown lesions). Two machine learning methods were compared for the indoor detection: neural network radial basis function (RBF), which is regarded as a powerful classifier for spectral reflectance data, and K-nearest neighbor (KNN). Overall, RBF performed better than KNN in different stages of the disease. The water index (WI) (Eq. 7) and anthocyanin reflectance index (ARI) (Eq. 8) and TCARI (Table 3) were the optimal indices for laboratory conditions and UAV-based diagnosis of infected trees, respectively. Identification accuracy of healthy and non-healthy trees from UAV-based detection was good, and the classification accuracy achieved for the late stage detection under laboratory conditions was 92%. But the authors also noted that immature fruit could not be used for early detection.

$$ {\text{WI}} = { }\frac{{{\text{R}}_{900} }}{{{\text{R}}_{970} }} $$

(7)

$$ {\text{ARI}} = \left( {\frac{1}{{{\text{R}}_{550} }}} \right) - \left( {\frac{1}{{{\text{R}}_{700} }}} \right) $$

(8)

Detection on the deadly disease HLB, for which the only treatment option is to cut down and remove the infected trees, has long been a research topic of great interest. A comparison between a UAV-based and an aircraft-based system for identifying HLB was conducted using multispectral imagery (Garcia-Ruiz et al. 2013). Compared to earlier case studies, the opportunities for using platforms with different spatial resolutions for disease classification were demonstrated. Resolutions of 0.5 m and 5.45 cm per pixel were employed for the aircraft and UAV, respectively. More indices were analyzed: 6 spectral bands and 7 vegetation indices. For the classification methods, support vector machine (SVM) with kernel performed better than linear SVM, linear discriminant analysis (LDA), and quadratic discriminant analysis (QDA). Results showed that the identification accuracy from the UAV was 67–85%, while aircraft-based yielded 61–74%. The authors suggested that future studies should focus on algorithm development, image acquisition, and the temporal effect of aerial identification of HLB. Similarly, a HLB detection accuracy of 81.75% was achieved when 16 vegetation indices were extracted for the classification based on SVM method (DadrasJavan et al. 2019). The study also found that the registration errors between bands of multispectral sensors could lead to a lower classification accuracy when the errors were larger than one pixel, especially for the application of UAV technology. In general, radiometric calibration complicates the process of HLB detection. However, the study showed insignificant effects of radiometric calibration on the discrimination of HLB-infected and healthy trees when the data were collected consistently with similar illumination and atmospheric condition (Pourazar et al. 2019). Disease scouting contributes to the control of disease. With the help of UAV-based platforms, the scouting efficiency was improved. However, UAV-based scouting is influenced by flying time, due to the limitation resulting from the payload.

In addition to disease identification, the main concerns investigated by researchers are early disease detection and severity evaluation, especially in the application of fruit breeding programs. A type of fungal disease, verticillium wilt (VW) in olive, greatly impacts the final yield and even leads to tree mortality. By analyzing thermal, multispectral, and hyperspectral datasets derived from UAV, (Calderon et al. 2013) aimed at the early detection of VW and discrimination among different VW severity levels. They analyzed different indices, e.g., physiological indices, and other indicators. Based on the finding that VW can cause water stress changes in olive trees, the investigation demonstrated that the reduction in ${\text{g}}_{{\text{s}}}$ was associated with an increase in PRI₅₇₀ and a decrease in fluorescence. Based on this, not only the early detection of VW was achieved, but also the discrimination of severity levels. Olive orchards with different agronomic characteristics were compared, which enhances the flexibility of the detection method proposed. The carotenoid reflectance index 2 (CRI2) and NDVI were also validated for detecting the early and advanced VW-infected trees (Iatrou et al. 2016). Apart from this, changes in the NDVI rate was found sensitive for monitoring the effects of plant growth enhancer formulation (PGEF) on the recovery of the trees, which can further enhance the management of VM in olive orchard.

Apple scab significantly affects the yield and quality of apple fruit and has become a major problem in apple orchard. It is caused by the Ascomycete fungus, Venturia Inaequalis. Research has shown the potential of making a risk evaluation model on the monitoring of apple scab in orchard using UAV technology (Stella et al. 2017). Apple scab can be indirectly monitored by acquiring leaf wetness data. On the basis of this, data extracted from UAV provided precise inputs to the evaluation model for the risk prediction based on the output of the model, the leaf wetness data. Detection methods based on thermal or spectral sensors make up for the shortcomings of classical detection methods that rely on visual observation by orchard experts. RGB sensors might also be used to detect diseases with apparent visual traits. Fire blight of apple is caused by the pathogen Erwinia amylovora infecting apple flowers, fruits, and the rootstock. Typical visual symptoms are the blackened shoots. This disease particularly threatens the production of commercial orchards (Salm & Geider, 2004). A recent study employed multispectral, hyperspectral, and RGB sensors to evaluate fire blight severity and found that detection from features derived from RGB and multispectral images was inferior to detection derived from hyperspectral images (Jarolmasjed et al. 2019). The index of normalized difference spectral indices, computed from hyperspectral datasets, showed moderate to high classification accuracy, ranging from 71 to 93%. Spectral bands between 710–2340 nm proved relevant for the classification.

Other applications

Pesticides can be used to support protection from pests, especially in the case of economic products, such as fruit trees. The usual management measure to deal with infestation with citrus leafminer (CLM) which threatens the production of citrus is chemical control (Qureshi et al. 2017). However, the uncontrolled and inappropriate use of pesticides affects biological systems, polluting preserved areas and damaging ecosystems. In addition, manual spraying exposes workers to a high-risk setting full of harmful chemicals. In the precision agriculture literature focusing on pesticide spraying systems in orchards, solutions based on UAVs are proposed to be safer, more precise, and more affordable than manual spraying or manned agricultural aircraft (Martinez-Guanter et al. 2019; Zhang et al. 2017). Nonetheless, aerial spraying can be inefficient in practice without a reasonable spraying strategy and detailed and precise information support, e.g., the identification of tree crown areas that are regarded as target spraying areas. Compared with other UAV-based management activities, there are more restricted operational parameters for sprayer UAVs, such as spray nozzle control, flying height, and speed. A deviation from the flight route or a change in wind direction will significantly impact on the droplet deposition distribution uniformity. Tree shape should also be taken into account even under the same spraying system (Zhang et al. 2016). For example, in the case of inverted triangle-shaped citrus trees, the lower layer was found to be the part with the most uniform distribution (CV = 32.44%), and they received higher droplet density than triangle-shaped trees (Tang et al. 2018). Other plant shapes, i.e., hedgerow and open-center-shaped, show different performance and optimal operation parameters. While the optimal control parameters of droplet density for the inverted triangle-shaped citrus tree was determined using Taguchi method, the spraying height was 1.4 m and the flight speed was 1.0 m/s (Hou et al. 2019). In terms of control effect against pests like CLM, a case study showed that a UAV-based system could achieve 65–75% of the control effect of manual spraying. But high efficiency and low cost of UAV-based spraying was observed at the same time (Zhang et al. 2017).

A limited number of earlier studies focus on intelligent and real-time application of sprayer UAVs which, with the help of machine learning and powerful computation support, have high potential for precision work. Using the mutual subspace method, an intelligent spraying UAV system could achieve an average recognition accuracy of 70% for spray or non-spray areas (Gao et al. 2019). This result falls within the requirements of precision agriculture, efficiently utilizing chemical inputs while reducing the environmental damage. However, the bottlenecks include limited battery capacity, the large volumes of liquid that must be carried, and the difficulty of reconciling high spraying speed and computation speed with promising recognition accuracy. For further study, deep learning methods should be tested against the machine learning-based recognition (Saldana Ochoa & Guo, 2019), in order to deal with the noise in the datasets and the negative impacts from the changing lighting conditions. Additionally, a pesticide spraying system based on multi-sensor data fusion algorithms may help bring about a high-efficiency revolution in the use of pesticides by accurately identifying and locating target trees and controlling pesticides.

Apart from research on sprayer UAV, several studies have used longitudinal UAV data for agro-environmental monitoring in orchards, such as the analysis of landslide evolution affecting the olive orchard (Fernandez et al. 2016) and the delineation of management zones for pest control (Mendez-Vazquez et al. 2019). Vegetation ground cover (VGC), the vegetation cover spontaneously grows on the surface of the ground, is an important component in the ecological system. A vegetation index derived from UAV, inverse ratio vegetation index (IRVI), was suggested to be the most sensitive index for the quantification of the density of VGC, and IRVI and ratio vegetation index (RVI) (Table 4) could most accurately distinguish the VGC densities > 80 in a cover interval range of 10% (p < 0.001) and VGC densities < 30% in a cover interval range of 15% (p < 0.01), respectively (Lima-Cueto et al. 2019). Monitoring tasks in orchards are different from other agricultural activities. Cost-effective monitoring approaches are always the classic problem not only for the growers in the orchard but also the researchers. On the basis of this, a study focus on the development of customized sensor capable to be mounted on the UAV show several advantages (Barrows & Bulanon, 2017). The customized low-cost multispectral sensor, for which the original internal infrared filter was replaced with a special dual-band filter, was proved to be comparable with the commercial grade sensor in the estimation of NDVI in orchards.

Table 4 Vegetation indices applied in the quantification of VGC

Fruit tree detection or classification is fundamental to the majority of UAV-based site-specific management in orchards. In general, machine learning and OBIA methods are currently the most popular methods for the detection and classification (Neupane et al. 2019). For instance, in a case study of agricultural resource management (Saldana Ochoa & Guo, 2019), a deep convolutional neural network (CNN) was employed. The processing chain proposed needs long-time and large datasets for training, which reveals the important need to strengthen the input requirements of CNN, although a significant classification accuracy was observed, with an F1 score of 0.89. Semantic segmentation may be a vital dimension in extracting crop classification from complex information. SVM performed well in the classification of both RGB and hyperspectral datasets. To segment citrus trees from the background in RGB images, an SVM model established by the calculation of 14 color features and 5 statistical texture features could result in a accuracy of 85.27 + _9.43% (Chen, Hou, et al., 2019). For the later case, misclassification of sunlit and shaded areas could be overcome with the help of SVM, achieving a classification accuracy of 94.5% of mango trees (Ishida et al. 2018). This classification capability can be further improved if the effects of wind on the data collection can be reduced, as wind deforms the flexible fruit-tree structure, thereby affecting image overlapping. Within the classification chain of a OBIA-based olive mapping study, dividing the original UAV image capturing a large plantation into subsets was suggested to speed up and facilitate the calibration (Karydas et al. 2017). Apart from these two popular methods, vegetation indices like NDRE, and the DSM could also be applied in fruit-crop discrimination (Handique et al. 2017). Most studies focus on the classification between fruit trees and other plantation while a combination of univariate and multivariate statistical approaches was applied for olive cultivar recognition. Results suggested that the classification accuracy between scions was 90.9%, however, 68.2% of the discrimination cases between rootstocks failed (Avola et al. 2019).

In addition to the standard applications discussed above, UAV-based management in the domains of regular crop monitoring and food quality tracking also have considerable impact on the development of UAVs in orchard management. In some cases, studies on these aspects have focused on fruit trees at a smaller scale than the standard orchard, or on fragmented land holdings. (Handique et al. 2017) reported on the collection of crop statistics in hilly terrain or terrace cultivation systems, such as in the northeast of India, where UAVs were used for crop discrimination for farming systems in hilly landscapes. The possibility of discriminating banana, orange, plum and bamboo with vegetation indices such as NDVI, NDRE and GNDVI was validated. Among countries located in the Pacific region, bananas, coconut and sweet potato are major food crops (Halavatau & Halavatau, 2001). However, high-risk disasters such as cyclones and storm surge occur here frequently and threaten food security. In this context, UAVs enable robust food assessments and the localizing of security and classification of diverse crops, especially of targeted fruits like mango, papaya, and coconut (Saldana Ochoa & Guo, 2019).

Discussion

In this survey, the research on UAV-assisted orchard management literature has been discussed in terms of five categories: fruit-crop resource efficiency, geometric traits, productivity, disease, and other applications. The first three categories account for 67% of the publications reviewed (Fig. 5), while the other two categories merit highlighting. Gaps and potential of evolving technologies in the domain of UAV-OM have been explored, ranging from application scenarios and UAV platform development to aerial data processing methodologies. In summary, UAV-based monitoring has very broad application prospects in orchard management because of its advantages related to flexibility, high efficiency, and low monitoring costs. In the following subsections, research gaps and opportunities in the near future are discussed.

Geometric traits of fruit trees

Different geometric traits of fruit trees have been measured using UAV imagery technology. To determine the state of art of the outputs, general geometric measurement performance was analyzed per year of article publication for four validation parameters: coefficient of determination (R²), classification accuracy, root mean square error (RMSE), and F1-score (Fig. 8). The general performance of the selected geometric traits assessment from UAV imagery improved significantly in the last two years surveyed: 2018 and 2019. The reasons could be the advancements in sensor precision and data processing methodologies. Nevertheless, algorithm development needs further improvement in case studies on, e.g., the estimation of crown diameter, which shows a low R². Within the analysis of estimation accuracy (Fig. 8), the latest accuracy achieved for fruit-tree detection and counting is 99.9%, thanks to the employment of machine learning and deep learning (Ampatzidis et al. 2019). The overall trend in development is positive, but the compatibility of models needs further testing against different fruit species.

Fig. 8

Comparison of performance for geometric trait estimation in the 22 articles on the application of fruit-tree geometric traits. CPA crown projection area, PPC plant projective cover. Note: If an article described the results with two validation dimensions, both were included in this figure

Studies over the past five years demonstrated the positive role of UAVs in fruit tree geometric measurement. To further enhance the adoption of UAVs for this application, the following developments can be identified:

• 3D representation: A realistic 3D representation of fruit trees is fundamental to the monitoring of geometric traits (Fig. 8). To deal with the problem of improving 3D reconstruction accuracy, investigation on the effect of flying speed, data capture view, GSD value and image overlapping parameter should be conducted, as demonstrated in earlier studies (Torres-Sanchez, Lopez-Granados, et al., 2018; Xue et al. 2019).

• Model generalization: As found in the literature on geometric measurements the validation of the algorithm was based on one fruit species only (Johansen et al. 2018; Mu et al. 2018), more universal models should be developed and tested. Fruit varieties, orchard areas, irregular or other planting patterns and various climatic conditions (Fig. 4) should be extended to validate the robustness of the algorithms proposed.

• Data processing efficiency: Time-consuming issue is intimately tied to ortho-mosaic image processing and DSM generation as shown in previous studies (Sun et al. 2019). Thus there is an urgent need to further develop SfM method and faster algorithm for DSM generation and ortho-mosaic processing.

• Automation: Current measurements are semi-automated (Marques et al. 2019), and manual interference even leads to subjective errors. Further experimentation into automatic fruit tree identification and geo-referencing is strongly recommended. More sampled trees and the employment of artificial intelligence, e.g., machine learning, would provide more definitive evidence in automated and simultaneous identification.

• LiDAR sensor: Most of the research used passive sensors, e.g., RGB and multispectral sensors (Fig. 6). Only one article used a laser scanner (Hadas et al. 2019), which significantly yielded an apple-trees identification accuracy of 99% (shown as the second case scatter point in 2019 in Fig. 8). The potential of LiDAR application in estimating fruit-tree geometry deserves special attention due to its advantages in point cloud analysis. In addition, comparison between RGB and LiDAR-based aerial geometric measurement would also be a fruitful topic.

Resource efficiency

Stem water potential ($\psi_{s}$), stomatal conductance (${\text{g}}_{{\text{s}}}$), and crop water stress index (CWSI) have been regarded as useful indicators for water status monitoring and irrigation strategy support in orchards (Ballester et al. 2018; Shackel et al. 1997). CWSI and alternative indices derived from UAV observations show promising results for water status assessment in orchards (Bulanon et al. 2016; Zhao et al. 2017). To date, a variety of indices have been compared against the commonly accepted indicators, $\psi_{s}$ and ${\text{g}}_{{\text{s}}}$, and the reliable indicator CWSI (Table 3).

The correlation between remote sensing indices derived from UAV imagery and the three indicators mentioned above was analyzed (Fig. 9) based on the validation results for the coefficient of determination (R²). In total, five fruit species, almond, citrus, apricot, peach and olive, were included. To clearly visualize the distribution of performance of each index, samples used for validation were divided into two categories, fruit species mixture and individual species. Because some research validated the performance of the method developed on an orchard with several fruit species instead of on a mono-species orchard (Park et al. 2017), mixed species was added to the analysis. On the other hand, the 11 studies on which Fig. 9 is based indicated that in UAV-based orchard management the tendency is to develop monitoring models or algorithms applicable for diverse fruit species. If an article compared indices against different species under different irrigation treatments, the best performance of each index toward a specific fruit species was selected for inclusion in Fig. 9. In addition, as some indices belong to the same spectral family and are of less interest, or rarely selected by researchers, the one in the same family with relatively high correlation was included in Fig. 9. Several data in the figure were obtained from the same experiment.

Fig. 9

Performance of diverse remote sensing indices for water status assessment in 11 articles: a Correlation with stem water potential ($\psi_{s}$) b Correlation with stomatal conductance (${\text{g}}_{{\text{s}}}$) The vegetation indices presented are described in Table 3

When it comes to the correlation with $\psi_{s}$ it was found that CWSI and the difference between Tc and Ta were generally relatively highly correlated with $\psi_{s}$ (Fig. 9). Some indices show high correlation with $\psi_{s}$ for specific fruit species, e.g., PRI_(570–515) and TCARI/OSAVI, while some indices level out at a low correlation, e.g., MTVI1 and Xanthophyll indices. An optimal estimation index for specific fruit species can be determined, e.g., the difference between Tc and Ta is the best for peach tree water status assessment. Similar to the analysis of $\psi_{s}$, the correlation between the indices derived from UAVs against ${\text{g}}_{{\text{s}}}$ showed that the most investigated indices were canopy NDVI, TCARI/OSAVI, and PRI, and that these indices performed differently for different species. TCARI/OSAVI and canopy NDVI showed relatively high correlation with ${\text{g}}_{{\text{s}}}$ for several fruit species, so have potential as a universal index to be employed in diverse orchards. The plant-based indicator CWSI is regarded as a reliable tool for irrigation strategy support (Gonzalez-Dugo et al. 2014). The correlation between remote sensing indices and CWSI was explored among five fruit species in relation to water status assessment. Canopy NDVI showed high correlation with CWSI in three fruit species, almond, apricot, and peach, while PRI generally showed a relatively low correlation in most fruit species. In orange species, all the indices showed low correlation with CWSI. It is very important to compare variable indices with CWSI in order to determine alternatives for water stress assessment. More fruit species and indices should be tested in the near future, to enable optimal indicators to be determined for specific fruits.

The majority of research focus on the resource efficiency in orchard, especially on the estimation of water status, has yield promising results. Further research should focus on the following scope:

• Model generalization: Most research were conducted under specific conditions, which makes the methods proposed low-range applications (Caruso et al. 2019; Park et al. 2017). To establish an accurate and reproducible model, methods (Fig. 9) should be established and validated in various conditions, e.g., different weather, crop phenological stages, crop fields and management conditions.

• Data calibration: Assessment of fruit tree water status mainly relies on temperature information derived from thermal images (Fig. 6). Yet, the reduction of effects from solar motion, atmospheric thermal path radiation and transmittance on the indices calculation is a problem (Berni et al. 2009b; Zhao et al. 2017). Ensuring consistent relationship analysis performance and high assessment accuracy, the optimization of calibration strategy and the management of thermal drift effect to thermal sensors are necessary.

• Intra-canopy variability of water stress: Current research focused on the estimation of water stress at tree scale (Ballester et al. 2018). Investigation of the variability at intra-canopy level is needed, especially for the analysis of fruit tree genotypic response to the water constraints.

• Periodic mapping: Previous studies developed their methods by doing UAV flight one time only (Jorge et al. 2019). By contrast, periodic mapping of water status in orchards can not only test and enhance the robustness of methods proposed but also facilitate the adoption of UAVs in irrigation management for growers.

Productivity and disease monitoring

Although most of the research focused on resource efficiency monitoring and geometric traits estimation, other aspects of orchard management were also covered, such as nutritional status and yield monitoring. For these aspects, crop physiological changes need to be understood and reliably modeled. Interdisciplinary investigation is a challenge for researchers in the domain of UAV-OM. UAVs are still generally underused and more comprehensive and in-depth exploration are needed. The research on the nutritional status monitoring of fruit trees shows an increasing trend, which is important not only for improving the output of fruit industry, but also for guiding the use of chemical fertilizers and pesticides. Yield estimation and prediction is then key for decision-making, especially for harvest strategy. As the costs of RGB sensors fall, the use of sensors to determine the optimal harvesting period for orchard production will increase. Yield estimation entails estimating flowering, which benefits the thinning activities. The complex structural traits of fruit crops may force UAVs to fly between tree rows in order to achieve an optimal inspection angle and a fine spatial resolution. Further investigation and experimentation are strongly recommended in the following aspects:

• Yield estimation at the tree or fruit level: A study applied ANN and yield an apple segmentation accuracy of 99.12% showed the great potential of UAV in fruit yield estimation, though the data was manually collected by emulating UAV capture conditions (Sabzi et al. 2018). Unfortunately, no research focused on yield estimation at fruit level was conducted (Fig. 7). Thus real-time direct estimation of fruits is encouraged.

• Method generalization: Current achievements in indirect yield estimation and health status monitoring are positive, as introduced in fruit-tree productivity traits section. However, there is abundant room for further progress in enhancing the robustness of the methods proposed (Fig. 7). Periodic mapping, as demonstrated by the study (Perry et al. 2018), is encouraged. Yet performance for different crops and growing stages remain unanswered at present.

• Machine learning: Methods and models proposed in some research (Fig. 7) need to be adjusted when apply them into a new case (Horton et al. 2017). Advanced machine learning algorithm can solve this problem, but also issues from the complexity of lighting intensity and conditions need to be taken into account.

In regular management, an outbreak of fruit-tree disease cannot be ignored. So far, the research on early warning monitoring of fruit-tree diseases has not shown an increasing trend, though a few studies have suggested the feasibility of UAVs in detecting biotic diseases in orchards. One reason may be that most fruit-tree diseases are not lethal, while HLB disease has attracted more attention because of its globally lethal effect on citrus crops (Arredondo Valdés et al. 2016). The complexity of pathological analysis for disease detection is also an important factor restricting related research. In other words, the development of UAV-OM for disease diagnosis is limited by laboratory detection or pathological research in particular cases. Future efforts could focus on the changes caused by disease, ranging from apparent external traits, such as color and texture, to fruit-tree photosynthesis. In addition, the substantially reduced spatial resolution of UAV is another issue can not be addressed and affect the disease monitoring. For example, wheat yellow rust can be detected in inoculation stage by analyzing the spectral reflectance differences, but it is difficult to monitor with the UAV though advanced sensor and low flight altitude, 16 m, were designed (Su et al. 2018). Further studies should focus as follows:

• Timely and localized diagnose models: Differing from other UAV applications in orchards (Fig. 7), disease monitoring is regional (Stella et al. 2017). Thus statistical study is encouraged to build timely, robust and localized disease detection models.

• Machine learning: Current studies extracted too many features for the disease monitoring (DadrasJavan et al. 2019), which makes the reduction of the features used the tendency. Though SVM, random forest classifier and ANN have been deployed (Fig. 7) and yielded promising results, advanced machine learning algorithms needed to be explored for improving disease detection efficiency and even monitoring the diseases undetectable currently.

Other applications

Inspection in orchards leads to the implementation of management operations such as pesticide or nutrient spraying using UAV sprayers (Tang et al. 2018). Currently, research is mainly exploring the selection of the spraying parameters, e.g., operation height, and the whole process is remotely controlled. Intelligent spraying requires the spraying system to automatically identify objects to be sprayed and to have automatic variable-spraying ability. On the basis of precise information sensed by sensor-equipped UAVs (Gao et al. 2019), operational systems could achieve precision spraying in orchards. The combination of remote sensing and automation deserves to be highlighted. Apart from this, fruit-tree classification and identification has been the subject of current research. Promising results become the basis for further investigation of fruit-tree at individual level. The UAVs developed for orchard management could be applied to related agricultural domains, e.g., the food supply chain (Saldana Ochoa & Guo, 2019). And the monitoring technology in orchards could also be used to improve the performance of agricultural monitoring on a large scale. Further research topics identified are the following:

• Spraying automation: Optimization of UAV control parameters for ideal pesticide spraying has been the subject of many research (Hou et al. 2019; Tang et al. 2018). However there is a still unanswered question about the effects of deviated flight routes caused by manual control. Thus validation with automatic spraying process is recommended, especially for facilitating real-time precision spraying.

• Statistical study: To develop robust fruit-tree classification methods, statistical studies will be needed for the isolation of error sources, thus determine the limitations of the proposed solutions (Ishida et al. 2018). On the other hand, optimal parameters for the implement of proposed methods can be highlighted, such as the optimal flying height during data acquisition.

• Deep learning: Machine learning and OBIA has been widely deployed in tree detection and classification (Karydas et al. 2017; Saldana Ochoa & Guo, 2019). Despite the promising results, further progress in improving the performance of proposed methods in various environmental and agronomic conditions with advanced deep learning algorithm need to be undertaken. And the first issue is the availability of larger training datasets with multiple fruit tree species.

• Automation: Automatic classification and identification of trees is still challenging. The complexity of orchard environment, e.g., the changing solar illumination, seasonal vegetation in high density modern orchard, makes semi-automated methods (Chen, Hou, et al., 2019) the optimal solution for current orchard management tasks. Studies with more focus on the automated and simultaneous classification are therefore recommended.

• UAV versus ground vehicles: The differences of imagery capture angle and spatial resolution between UAV and ground vehicle significantly effects their performance in orchard management (Zhang et al. 2019). Combining UAV and ground vehicle, or even other platforms (Table 1), could be an advantage, in different applications.

• UAV customization: Current commercial UAVs can not meet all the requirements in orchard management. Customization of sensor deployed or UAV system provides new insights in the adoptability of UAVs, as demonstrated in one case study (Barrows & Bulanon, 2017). For example, a UAV system with the capability of upward image acquisition or acquiring datasets at specific fruit-tree organs scale will make unique spatial resolution or details of fruit-tree structure available for many research.

UAV platforms

Almost all the research in UAV-OM employed commercial UAVs due to the cost-effectiveness compared to handcrafted or industrial UAVs. Multi-rotor UAVs is the most widely used. While the complicated operation of fixed-wing UAVs make it less popular, such as the requirement of minimum flight speed before they stall. UAVs with VTOL system have emerged as new powerful platform. Its freedom from site condition restrictions make it capable work even on steep orchards (Torres-Sanchez, Lopez-Granados, et al., 2018). RTK GNSS is becoming standard resulting in increased geometrical quality in SfM processing (Xue et al. 2019) and derived products like ortho-mosaic and 3D point clouds (Marques et al. 2019) of fruit trees in orchard. This would allow comparison of changes in geometrical properties of trees within growing season and over years as indicator for productivity. The fundamental constraint to developing technology applications is the hardware system. The current limitations to the adoption of UAV platforms to orchards are the payload and endurance (Garcia-Ruiz et al. 2013), especially for UAV-based pesticide sprayers and LiDAR UAVs. Increasing the battery capacities increases the payload, yet the payload of UAVs significantly affects the endurance performance. With the improvement of capacity of batteries, all types of UAVs flight duration will be extended, and VTOL will be more suitable for the mapping task in large surface fruit orchards and plantations. Powerful UAVs capable of carrying multiple sensing systems are generally more costly and not affordable for applications, especially in developing countries. Data collection opportunities and timing in orchards are limited and restricted. To ensure resistant flight in various weather and environmental conditions, the design of UAVs with weather-proofing capability is needed. In addition, platform vibration affects the accuracy of aerial indices and image quality. This could be resolved by improving UAV design and the post data-processing procedure. When deployed optimally in orchard management, UAVs are currently operated by persons with the skills of professional pilots. In the near future, the human-UAV interaction and ease of operation are excepted to be further improved.

The articles reviewed revealed a shortage of customized UAV platforms. Various outdoor agricultural operations are very complex. For investigations on UAV-OM, requirements related to aspects such as image capture parameters and sensor automatic adjustment differ from each other. To set up datasets of optimal quality, researchers should clearly understand the theory underlying the data collection systems. The first basic step is to assess the suitability of the platform employed and whether it can be customized to meet the unique demands for the specific problem to be addressed. Although platform customization has drawn attention in studies on water status assessment and spraying (Ortega-Farías et al. 2016), it should also be considered in other scenario.

Despite UAVs have been proved effective in assisting growers for orchard management, current development of UAVs is still far from meeting the requirements of precision agriculture in orchards. Futuristic development of UAVs should cover the following heads:

• LiDAR UAVs: UAVs with RGB, thermal, multispectral, and hyperspectral sensors have been explored in orchard management (Fig. 6), yet UAVs with LiDAR are undervalued. The weight of LiDAR sensors and their high power consumption pose a challenge for LiDAR UAVs, though significant advantages of it in geometric traits measurement has been demonstrated (Hadas et al. 2019).

• Onboard processing: By reducing the requirements of network bandwidth, commercial UAVs with onboard processing capabilities may benefit its controllability. On the other hand, with the advent of the 5G era, it would also enable the efficient communication among UAVs and other platforms which are used to execute management operations. In this way, so-called swarm intelligence (SI) can be achieved in orchards where detection and actuation is divided among different platforms (Zhang et al. 2019).

• Active management platform: Current function of UAVs are mainly imaging and sensing (Fig. 6). However, more active involvement in orchards is required. Besides the UAVs with recognition capability for automatic and precise spraying (Gao et al. 2019), development in active visual scouting in orchards, e.g., searching for the trees with low nutritional status and even pests, will also be of interest.

• Obstacle avoidance: The complexity of horticultural environment limits the application of UAVs in orchard. Compared with conventional remote sensing, UAV automatic scouting between rows of fruit trees has potential of collecting data with higher resolution and yielding better performance (Das et al. 2015). To deal with this, UAVs with accurate obstacle avoidance system are needed to lessen the threat from trees and even birds to the flight safety.

• Night vision: Previously published studies have shown the feasibility of UAVs for monitoring various fruit tree traits that are directly related to the aerial imagery or spectral information and achieved promising results. Yet, for the complex traits that are indirectly related, e.g., the aerial indicator of Fusarium wilt of banana, few achievements were reported. UAVs with night working model may provide new insights to UAV-OM. Additionally, experiments from ground vehicles also indicated the potential of UAVs with night vision in improving the yield estimation accuracy (Chen et al. 2017; Wang et al. 2018).

Sensor payload

Promising performance for various applications has been shown for five types of sensors that can be used on UAVs, namely RGB, thermal, multispectral, and hyperspectral sensors, and LiDAR. Based on the analysis of the sensors deployed, further analysis of the development of various sensors in different management applications was conducted (Table 5). Besides further explorations within orchard management, the blank area indicates potential for a “new” sensor more capable in a specific domain, such as utilizing LiDAR in yield estimation. All five sensors need to be corrected or calibrated within the process of aerial data processing in order to improve the monitoring accuracy. However, current RGB sensors lack the function of camera calibration and radiometric calibration, which can affect the sensor performance, especially for geometric estimation. Meanwhile the impacts of filters, optical defects, and sensor lenses, and even the selection of radiometric correction approaches for calculating the accuracy of remote sensing indices should also be kept in mind (Tu et al. 2018). Hyperspectral sensors gained the attention of the scholars in UAV-OM. Besides the reason of its size and less affordable characteristic, the geometric calibration accuracy also seriously affect the popularity. Though the sensor is capable to provide centimeter level spatial resolution, the geometric calibration accuracy is difficult to reach the same level. The application of LiDAR for fruit orchard management is relatively unexplored (Díaz-Varela et al. 2015); upcoming research areas may demonstrate the potential of LiDAR in orchard management, especially for geometric traits measurement. Regarding the pros and cons of each sensor type, the tendency is for imagery or information fusion (Delalieux et al. 2014; Kestur et al. 2018). The basic principle of this multi-modal sensing approaches is combing different sensos on the same UAV platform. Datasets collected from different sensors contain unique fruit-tree traits, and imagery fusion can achieve the goals of getting these traits to complement each other and improving the detection accuracy. The development of UAV platforms is the basis of information fusion technology.

Table 5 Potential of UAV-based monitoring sensors for diverse orchard management activities

As the basis of the adoptability of UAVs in orchard management, current limitations and perspectives of sensors deployed are as follows:

• Development in size and weight: The selection and cost of UAVs are limited by the size and the weight of sensors to be deployed e.g., the LiDAR sensor has more weight, which makes the UAV-derived LiDAR data largely unexplored (Hadas et al. 2019). What is now needed is the further miniaturization of sensors, especially for hyperspectral sensors. On the other hand, it is also intimately tied to the application of multi-modal sensing approaches.

• Development in imaging capability: The existing sensors limits the quantitative investigations due to the difficulty of extracting quantitative information from the data collected. Advanced sensors capable to obtain quantitative information in complex orchard environments are needed.

• Integration of sensor and UAV control systems: The vibration of UAVs during data acquisition process affect the performance of sensors payload, e.g., the geometric distortion occurs in hyperspectral data (Vanbrabant et al. 2019). The integration of the two control systems enable the dynamical control of the imaging sensors and further improve the data quality.

Data collection strategy

There are potential research areas in the design of fieldwork strategy. In some areas, continuous cloud cover greatly restricts the use of satellites for most of the year, making the potential of alternative sensing platforms like UAVs attractive. However, the aerial surveys should be postponed in windy conditions or light rain. Next, the trade-off between flying altitude and required image resolution deserves attention. A higher flying altitude produces lower spatial resolution. The differences of spatial resolution may affect the validation of the method developed. As a satellite maintains a constant altitude, effects of this are rarely observed. Many investigations produced a suitable solution for a specific problem by employing a strict set of operational parameters, e.g., flying height and angle of capture. Next, the effects of these parameters on the estimation accuracy should be studied, in order to expand the practical scope of approaches developed and to determine the optimal setting for the operational parameters.

Most research focused on specific fruit species at a certain growing stage under certain conditions. In other words, the achievements were attained under specific circumstances. The ideal situation is to develop approaches capable of monitoring various species or different growing stages of the same fruit species. Comparison between different species or certain growing stages provides a better algorithms validation strategy. Subsequently, various training systems or planting patterns could also be tested.

Development of methodology

Compared with conventional direct measurements, UAV-OM requires empirical statistics, reverse modeling, and image- processing technologies for effective, automatic, and precise management in orchards (Ballester et al. 2018; Saldana Ochoa & Guo, 2019). In the general workflow for UAV data processing, key steps are geometric correction and radiometric calibration. To deal with effects like solar motion, further calibration study is needed. Many monitoring methods have been proposed as being usable for similar case studies (Chen et al. 2017). However, the outdoor environment is far more complex than the indoor environment. Thus, problems in outdoor practice demand more sophisticated solutions. It is not surprising that existing algorithms are being applied to deal with certain problems, but researchers tend to overlook the need to refine the algorithms. This also explains why the developed algorithms prove unsuitable when tested under irregular conditions and why the classification performance is easily affected by background objects with characteristics similar to those of the target objects. Moreover, techniques like machine learning and deep learning have not been properly employed in UAV-OM. Focusing on the theory underlying the tools applied can overcome the limitations resulting from established techniques, especially in the case of application in a real agricultural environment. Finally, using large UAV datasets, efforts should be made to improve the efficiency of image processing (Saldana Ochoa & Guo, 2019). And statistical studies aimed at verifying the robustness of methods developed deserve more attention.

Conclusion

This review has provided an overview of different applications employing UAVs with multiple sensors for fruit orchard management. The majority of the research was conducted in the past 4 years. UAVs generally yield a fine monitoring efficiency and accuracy, which indicates their potential as novel remote sensing platforms. Yet, UAVs currently are mainly used by experts and there is still a need to make this technology directly benefit the crop producers providing them with precise information on the operational application of UAV technology in day-to-day operations.

UAV-OM investigation is yet in its infancy. The applications for resource efficiency and geometric traits are relatively mature, while yield estimation, especially the estimation at fruit level, disease monitoring and UAV-based sprayer will becoming increasingly important area. Results have demonstrated high correlations between various UAV-derived indices and target physiological traits measured manually. Yet new indices correlated to the complex traits which are difficult to directly assess remain undiscovered at present. Model generalization, data processing efficiency and automation are still challenging. Further studies, which take these three issues into account, will need to be undertaken. As a next step, the combination of artificial intelligence and remote sensing sciences will be able to close the gap between current research and precision orchard management. UAVs have promising application prospects in precision orchard management because of their fast and efficient monitoring. Real-time monitoring is the key trait of UAV-based remote sensing that makes up for the long periodic intervals of satellite monitoring. Timely fruit-crop-growing information like this will enable healthy crop growth to be assured and economic loss to be avoided. Further, growers may obtain real-time growth information from web or mobile applications using cloud computing and wireless transmission technology (Salamí et al. 2019).

In recent years, multi-rotor UAV is the most widely used UAV in orchard management and the majority is the commercial UAV. In the trend of continuous miniaturization of sensors, the limitation to UAV-OM is mainly the flying time due to current state-of-the-art in battery capacity. Different types of UAV sensors have their own place for specific monitoring activities but share the pros and cons. Thus, multi-sensor data fusion could be promising although was not yet investigated. In the case of LiDAR, despite its notable advantages for measuring geometric parameters, it is not commonly exploited due to its significant operational costs and the limited UAV flight time (Friedli et al. 2016; Garcia-Ruiz et al. 2013). Additionally, pre-flight flying parameter settings, such as UAV speed and field of view, affect the monitoring performance. Thus, statistical study to determine the optimal data acquisition parameters and understand the effects for specific research is encouraged.

Global orchard production is especially significant at regional scale. This diversity is attributed to the local climate and soil, geographic and topographic conditions, and high number of fruit species. The lack of publicly available datasets requires researchers to develop their own datasets although it could be more efficient to reuse images already acquired. At some point, growers will be encouraged to share the data gathered with their own UAVs and in situ observations to boost thee advances in UAV-OM.