Seamless integration of above- and under-canopy unmanned aerial vehicle laser scanning for forest investigation

Abstract

Background

Current automated forest investigation is facing a dilemma over how to achieve high tree- and plot-level completeness while maintaining a high cost and labor efficiency. This study tackles the challenge by exploring a new concept that enables an efficient fusion of aerial and terrestrial perspectives for digitizing and characterizing individual trees in forests through an Unmanned Aerial Vehicle (UAV) that flies above and under canopies in a single operation. The advantage of such concept is that the aerial perspective from the above-canopy UAV and the terrestrial perspective from the under-canopy UAV can be seamlessly integrated in one flight, thus grants the access to simultaneous high completeness, high efficiency, and low cost.

Results

In the experiment, an approximately 0.5 ha forest was covered in ca. 10 min from takeoff to landing. The GNSS-IMU based positioning supports a geometric accuracy of the produced point cloud that is equivalent to that of the mobile mapping systems, which leads to a 2–4 cm RMSE of the diameter at the breast height estimates, and a 4–7 cm RMSE of the stem curve estimates.

Conclusions

Results of the experiment suggested that the integrated flight is capable of combining the high completeness of upper canopies from the above-canopy perspective and the high completeness of stems from the terrestrial perspective. Thus, it is a solution to combine the advantages of the terrestrial static, the mobile, and the above-canopy UAV observations, which is a promising step forward to achieve a fully autonomous in situ forest inventory. Future studies should be aimed to further improve the platform positioning, and to automatize the UAV operation.

Introduction

Precise knowledge of the distribution of tree size, species, health, and growth is essential to all decisions that are relevant to forest ecosystems, ranging from the forest resource management to the protection of climate and biodiversity. Tree by tree (tree-wise) measurements, which are typically carried out in situ in forest sample plots, provide the fundamental reference data for all types of upscaling approaches for regional, national, and global level assessments. The correctness and completeness of in situ observations in sample plots determines the reliability of the reference, and further determines the credibility of forest-attribute estimates in large areas based on them. Meanwhile, the efficiency and the cost of in situ tree-wise measurements determine the spatial and temporal resolution of the reference, that is, the sufficiency of reference data. Therefore, correctness, completeness, efficiency and cost are equivalently important for the reference data collection, which directly determines the correctness of the decisions and policies that are reached, and is equally vital for forest owners, wood industries, ecological and environmental scientists, and governmental decision-makers in the field of bioeconomy, sustainability, and the climate and biodiversity protection.

Conventionally, a manual tree by tree measurement in field was understood as the most trustworthy method of acquiring tree parameters at plot-levels. The conventional methods are operational applied in forest in situ mensuration at all levels, despite of its well-recognized limits on the high labor and time costs (Návar 2010). Lately, manual field measurements start to face other challenges. Possible systematic errors are found in some of the most important tree attributes using conventional methods, e.g., in tree height estimates (Wang et al. 2019a; Jurjević et al. 2020). These limitations and challenges inspired new advancements in the forest field data collection through remote sensing technologies.

In the last 20 years, the introduction of Light Detection and Ranging (LiDAR), a.k.a. Laser Scanning (LS) systems integrated on various airborne and terrestrial platforms achieved a great success. A road map of LS-based forest investigation is currently being formulated, which uses the terrestrial laser scanning (TLS) and/or Mobile Laser Scanning (MLS) to collect plot-level field data (Liang et al. 2016; Wallace et al. 2017; Saarinen et al. 2017; Hyyppä et al. 2020b; Calders et al. 2020), and to further calibrate airborne laser scanning (ALS) data as well as other aerial and satellite data for regional and national level assimilation (Coomes et al. 2017; Urbazaev et al. 2018; Dalponte et al. 2019). However, barriers still exist for the automated systems to be independent from the conventional field measurements as a supplier of in situ reference information. More specifically, the challenges are stemmed mainly from the quality of the data collected and information provided (Wang et al. 2019b), e.g., the completeness and the geometrical accuracy.

One most significant challenge that all state-of-the-art LS systems are commonly confronted with is the omnipresent occlusions in forest environments. The occlusions from the crowns and stems largely reduced the completeness of the collected data on tree- and plot-levels. For example, in boreal forest environments, only approx. 50%–90% individual trees on a plot-level could be recorded using a typical multi-scan TLS data acquisition using five stations (one at the center and four at corners), which marks the highest completeness among all other LS systems including ALS and MLS (Wang et al. 2019b). High density aerial point cloud acquired using unmanned aerial vehicle (UAV) laser scanning system (ULS), for example, with about 450 points∙m^− 2 point density, could cover averagely 61% trees on a plot level due to occlusions of the intermediate and the suppressed trees (Wang et al. 2019b).

The occlusions of different tree parts reduces the tree-level completeness in point clouds, which significantly influences the accuracy of estimated tree attributes. For example, both TLS and MLS generally underestimated the tree height, especially when a tree was taller than 20 m, due to incomplete digitization of tree crowns (Liang et al. 2018). MLS has the potential to reduce the occlusion effects but is still under development, e.g., (Kukko et al. 2017; Shao et al. 2020a, b). High density ALS and ULS could provide reliable tree height estimates, however, the stem parameters such as the diameter at breast height (DBH) or the stem curve derived were problematic because of the insufficient or unreliable stem information in data (Liang et al. 2019). The limitations of existing systems significantly hindered the practical practices of automated forest observations.

Existing solutions for improving the completeness of data can be divided into two groups: the first is to densify the data collection such as to add multiple viewing angles of scanner in order to enhance canopy and stem completeness, e.g., (Roşca et al. 2018; Wu et al. 2020), or to densify trajectories in order to record more trees from more viewing perspectives, e.g., (Morsdorf et al. 2017; Del Perugia et al. 2019; Kuželka et al. 2020); The second is the fusion of datasets from different platforms, especially between terrestrial and aerial platforms, e.g., (Paris et al. 2017; Giannetti et al. 2018; Dai et al. 2019; Pyörälä et al. 2019).

Nevertheless, operations such as adding viewing angles and densifying scanning locations or trajectories generally sacrificed the efficiency and increased the cost of data acquisition. Fusion of terrestrial and aerial point clouds faced additional challenges such as the lack of availability of both terrestrial and aerial datasets, the low rate of the existence of correspondences in both datasets due to the occlusion effects, and more importantly the possible time lag between different datasets due to practical constrains which leads to information gaps.

In short, current solutions is facing a dilemma over how to achieve high tree- and plot-level completeness while maintaining a high cost and labor efficiency. New solutions are required to balance the trade-offs between these contradictory interests.

Recently, UAVs became exceedingly popular as platforms for close range remote sensing because of its high mobility and accessibility. Above canopy ULS and under canopy ULS were implemented and studied separately in existing applications. According to (Bruggisser et al. 2020), half of the stem circumference is required in above canopy ULS data in order to achieve reliable DBH estimates. However, to capture half of the stem circumference for all standing trees in forest using above canopy ULS is a demanding task (Liang et al. 2019). The feasibility of increasing stem visibility in above-canopy ULS using a low flight height (e.g., below 100 m above the ground) was tested, and was reported to be capable to provide DBH or volume estimates for big (DBH > 20 cm) trees or big branches (branch diameter ≥ 30 cm) with a comparable accuracy as that of the TLS systems in a few cases, e.g., (Brede et al. 2017, 2019; Wieser et al. 2017). These results indicated that the above canopy ULS is capable to provide stem parameters to certain extent, but was limited to sparse and matured forest stand.

An early under canopy ULS was reported (Chisholm et al. 2013), in which 73% of trees larger than DBH 20 cm within 3 m of the trajectory can be extracted, and the DBH estimates from the data has a relative root-mean-squared error (RMSE) of 25.1%. Recently, with support of simultaneous localization and mapping (SLAM), under canopy ULS was reported to capable to facilitate 93% stem detection rates, 0.60 cm (2.2%) RMSE for DBH estimates, and 1.2 cm (5.0%) RMSE for stem curve estimates in sparse forest stands (Hyyppä et al. 2020a). Other under canopy UAV approaches, e.g., (Krisanski et al. 2018; Kuželka and Surový 2018) used structure from motion (SfM) technology to generate photogrammetric point cloud for estimating stem parameters, and the accuracy was comparable to that of the terrestrial SfM approaches reported (Liang et al. 2015; Mokroš et al. 2018). In general, SfM faces challenges in tackling occlusion problems in forests, and is more difficult to implement in forest environments than the LS.

Considering the high mobility of the UAV platforms and the encouraging outcomes from the existing studies, a concept of integrating above- and under-canopy ULS data acquisition to efficiently improve the tree- and plot-level completeness was developed in this study. The approach is referred as “inside canopy ULS” in the following contents, which integrates the above- and under-canopy flights of a ULS system in a single operation. Such concept provides a new approach for the fusion of aerial and terrestrial perspectives and their advantageous using one single ULS system.

The aim of this study is to find answers of the following questions: 1) Is it feasible to generate an integrated above- and under-canopy point cloud using a single ULS system that follows an integrated above- and under-canopy trajectory in a single flight mission in forests; 2) Would a basic hardware system setup, namely, an ULS system supported by the Global Navigation Satellite System (GNSS) and the Inertial Measurement Unit (IMU) for platform positioning and point cloud geo-referencing, be sufficient for generating an integrated above- and under-canopy point cloud; 3) Would such inside canopy ULS promote plot- and tree-level completeness and how efficient would be the system; 4) How accurate would be the estimates of tree parameters (e.g., tree location, tree height, DBH, stem curve) based on such an integrated flight.

By answering those questions, in this study, a baseline of the performance of such inside canopy ULS for forest investigation would be marked, and the further research questions to enhance such systems would be clarified.

Material and experiment

Test site

The experiment was carried out in a 120 m × 120 m heterogeneous plot in a boreal forest located at Kirkkonummi, Finland (60°09′ N, 24°30′ E), as shown in Fig. 1. The main tree species comprised the Silver birch (Betula pendula Roth), the Norway spruces (Picea abies (L.) H. Karst) and the Scots pine (Pinus sylvestris L.). Trees in the plot were at various ages from new regenerations to matured trees, and the stem density was about 200 stems∙ha^− 1. The average tree height is 17.85 m, with a standard deviation of 8.83 m. The average DBH is 25.41 cm, with a standard deviation of 12.72 cm. Such stand condition represents a typical open boreal forest or a common urban area forest that is growing naturally. Compare to other reported ULS studies, this is a relatively young stand with plenty small trees whose DBH are less than 20 cm.

Fig. 1

The study area (120 m × 120 m) illustrated by the co-registered multi-scan TLS point cloud

Inside canopy ULS system, trajectory and point cloud

The flight was carried out using a Tarot 960 hexacopter (Tarot Aviation Technology Co., LTD, Wenzhou, China), which has a payload capability of 4 kg and a maximum flight time of 15 min. The onboard payload consisted of a Riegl miniVUX laser scanner (RIEGL Laser Measurement Systems GmbH, Horn, Austria), a Novatel SPAN-IGM-S1 GNSS-IMU (NovAtel Inc., Calgary, Alberta, Canada) and a Sony A7R camera (SONY Corporation, Minato, Tokyo, Japan).

The laser scanner, the GNSS-IMU, and the camera were mounted rigidly together and suspended under the UAV frame using vibration dampening elements. The GNSS-IMU and the laser scanner were mounted at an angle of approximately 18 degrees backward with respect to the UAV frame normal. This allows the laser scanner to measure the tree trunks with a slight angle from the side instead of directly looking downwards during the above canopy flight. Figure 2 illustrates the ULS system used in this experiment.

Fig. 2

The ULS system, including the drone, the scanner, and the GNSS-IMU

The flight trajectory was planned in advance based on the TLS data of the plot. The flight operation was carried out by an operator using a manual remote control. The operator followed the UAV to guarantee the visibility of the UAV. The flight started with several flight lines above the targeted area at an approximate 50 m height above the ground; then the UAV was lowered to a ca. 1.5-m height above the ground through a scouted canopy gap, and continued to fly under the forest canopy at 1.5 m above the ground until the end of the flight. Along the under canopy trajectory, the UAV was set to a hover mode and was rotated around its vertical axis for approximately 360° at any location that was feasible for such an operation. Because the scanner provides 360° degree cross-track 2D scanning profiles, this hover-rotation mode was applied to simulate a TLS scanning and to allow better digitization of the surrounding stems. After each hover-rotation, the flight continued along the planned open corridors below the canopy where there were no low hanging branches or high understory vegetation, until the UAV completely traversed the planed trajectories. No reference targets were used during the experiment. The whole flight operation, i.e., from the UAV takeoff to its landing, lasted for ca.10 min, and was carried out by one operator.

Visibility to GNSS satellites was maintained throughout the operation. The minimum number of tracked satellites was three. Such occasions of low number of tracked satellites happened for four times during the under-canopy operation, and each time lasted for a few seconds (less than 5 seconds). The maximum number of tracked satellites was 13 during the above-canopy operation, and the average number of tracked satellites were ten in most of the time during the operation.

The trajectory of the whole flight was calculated using the GNSS-IMU observations and Virtual Reference Station data provided by Trimnet web GNSS service (GEOTRIM, Vantaa, Finland) with base station coordinates and data commutated for the flight site. The trajectory was computed with NovAtel Inertial Explorer software (NovAtel Inc., Calgary, Alberta, Canada) using tightly coupled solution. The raw laser data were converted into a point cloud using RiProcess software (RIEGL, Horn, Austria). In the preprocessing phase, data with reflectance less than − 16 dB were filtered out as noise. Planar surfaces present in the point cloud were searched and used to calibrate the sensor orientation, i.e. boresight misalignments, with respect to the IMU.

Translations between the scanner and the positioning sensors were physically measured from the device compound to millimeter accuracy. A robust adjustment was applied for estimating the roll, pitch, and heading calibration values, and subsequently applied to the point cloud data. As this only solves systematic angular discrepancies, an additional step was taken with a RiPrecision optimization tool to solve the remaining dynamic errors in the trajectory, and to reconstruct the point cloud as accurately as possible. The generation of the point cloud took a similar amount of working time as the co-registration of the multi-scan TLS data. Figure 3 illustrates the trajectory with the final point cloud as a background.

Fig. 3

The trajectory (yellow and white lines) and the point cloud (green for vegetation and brown for ground) of the inside-canopy-ULS flight. Top left, bird view of the trajectory and the point cloud. The locations of hover-rotation mode of the UAV are indicated by the areas with high brightness in the point cloud. Top right, side view of the trajectory and the point cloud. Bottom, inside forest view of the trajectory and the point clouds

Reference data

The forest was scanned using TLS to collect the field reference data. A Leica ScanStation P40 scanner (Leica Geosystems, St. Gallen, Switzerland) was used. The scanner was set at the ‘Speed’ mode and the ‘Normal’ sensitivity level, which provides a 3.1-mm point spacing on both horizontal and vertical directions at a 10-m distance from the scanner. A full-field-of-view scan took 3.5 min. A multi-scan approach was taken and a total of nine scans were carried out in the plot. To support the co-registration of the scans, 14 locations were set up in the plot to place reference targets. For each scan, at least three spheres were visible in a neighboring scan. Sphere locations were measured by a Topcon HIPER HR GNSS receiver (Topcon Positioning Systems Inc., Livermore, United States) for geo-reference. The scan co-registration was accomplished using Leica Cyclone 9.1 (Leica Geosystems, St. Gallen, Switzerland) with a mean absolute registration error of 0.4 cm. The collection of multi-scan TLS data took two persons half a day to complete, and the co-registration took a few hours for one person.

The reference data were manually measured from the multi-scan TLS data. Ground points were first classified using the classification routine in TerraScan software (TerraSolid oy, Helsinki, Finland).

Tree stems were visually identified and extracted from the normalized point cloud data. All trees with a DBH larger than 5 cm were included in the reference. The total number of reference trees in the test area was 203. To derive the stem curve of a tree, stem diameters were measured at 0.65, 1.3, 2 m, then every one meter above ground level, until the maximum measurable height. The definition of height intervals of the stem curve was based on the generally applied practical rules of stem curve measurement for forest management in Finland. At each measurement location of the tree, the normalized point cloud was sliced with 10-cm thickness (i.e., 5 cm above and 5 cm below), and a circle was manually fitted to the sliced stem points using the TerraScan software. The center coordinates and diameter of the fitted circle were used to define the stem location and diameter of a tree at the corresponding height.

Tree heights were manually measured from the normalized point cloud for the manually detected trees. For each stem, the points corresponding to its treetop were visually identified in the point cloud and used to define the tree height. This means that the reference tree height in this study was derived from the multi-scan TLS data. Results from (Wang et al. 2019a) showed that in easy forest stands (ca. 600 stems∙ha^− 1), multi-scan TLS-acquired tree height has a high consistency with the field measured tree height at least until trees were below 20 m high. In the test site of this study, the stem density was 200 stems∙ha^− 1, the average tree height was 17.85 m, and approximately 75% of trees were below 20 m. The test site was largely open, which provided high visibility of surrounding treetops to each single scanning position. Therefore, the digitization of treetops in the multi-scan TLS data was guaranteed, and the manually measured tree heights from the multi-scan TLS are considered as reliable as the field measured tree height to be used as references.

Methods and results

The performance of the inside canopy ULS is evaluated through the map of individual trees and the estimation of tree parameters using the acquired data. To differentiate the errors originated from the data and that from the applied automated method, both manual and automated methods were used for tree detection. The completeness of tree detection is understood as an indicator for the plot-level completeness of the experimental inside canopy ULS. The accuracy of estimated tree parameters such as tree position, tree height, DBH, and stem curve are regarded as indicators for the tree-level completeness and the geometric accuracy of the inside canopy ULS.

The results are reported accordingly in two groups, e.g., the detection and the modeling. In order to evaluate the performance of the joint above-under-canopy flight in details, a subset of the study area is identified according to the operation and observation and trees in the plot are grouped according to the tree size criteria.

Automated stem detection and modeling

The inside canopy ULS data were processed in a similar manner to that of the above-canopy ULS data (Liang et al. 2019). The data processing began with the extraction of ground points. The original point cloud was allocated in a 2D grid of a 20 cm × 20 cm resolution, and the largest connected group of lowest points in each cell of the grid was selected as ground points. A DTM was generated using linear interpolation of the ground points. The point cloud was normalized with respect to the DTM, and the point at the top canopy (i.e., 20%) was removed to optimize the following stem modeling procedure. After the stem detection, the estimation of tree height was carried out using the original point cloud to minimize the influence of the DTM to the tree height estimates.

Stem points were identified through the analysis of point distribution in each point’s neighborhood. A tree stem was modeled as 3D cylinders to represent the stem’s shape and growth direction. The DBH and tree location were estimated from the model element at the 1.3 m breast height. The stem curve was estimated from the diameters at predefined heights, i.e., 0.65, 1.3, 2 and 3 m, to the maximum measurable stem height from the point cloud data. The tree height was estimated separately for large and small trees, according to a DBH threshold of 15 cm. For large trees, the highest point around the stem position was understood as the tree top. For small trees, which were often covered by large trees, the largest connected point group around the tree stem was first identified, and the highest point of the group was estimated as the treetop. In both cases, the tree height estimate was the elevation difference between the treetop and the lowest point of the extracted stem.

Manual stem detection

Besides the reference information derived from multi-scan TLS data, manual stem detection from the inside canopy ULS data was also employed to support the evaluations. The manual stem detection was carried out by an operator guided by the reference information. For each tree on the reference map, the visibility of its stem in the inside-canopy-ULS data was verified. A tree was regarded as omitted in data if the operator could not see its stem, or the operator considered that the recorded portion of the stem in data cannot support the estimation of its DBH, namely, no arc or circular shape can be recognized by human eyes from the (x, y) projection of the stem points. The manual stem detection result is regarded as preliminary evaluation of the quality of the data, which indicates the proportion of the individual trees digitized to a level that at least their DBH can be estimated.

The method of evaluation

The result of automated stem detection was compared with that of the manual stem detection and the reference data. The accuracy of stem detection was evaluated using the indicators including completeness and correctness. The completeness is the ratio between the number of correctly detected trees and the total number of reference trees. The correctness is the ratio between the number of correctly detected trees and the total number of detected trees. In addition, the ratio between the numbers of automated and manually detected trees was used to evaluate the performance of automated processing more accurately given the inside canopy ULS data.

The accuracy of the automatically estimated DBH and tree height was evaluated using the RMSE and bias, as well as relative RMSE (RMSE%) and relative bias (bias%). For the stem locations, only RMSE was calculated. For an extracted stem curve, the diameters at different tree heights of a tree were evaluated using the tree-wise RMSE_tw, RMSE_tw%, bias_tw and biast_w%.

$$ \mathrm{RMS}{\mathrm{E}}_{\mathrm{tw}}=\sqrt{\frac{1}{m}{\sum}_{i=1}^m{\left({d}_{h_i}^{\mathrm{ext}}-{d}_{h_i}^{\mathrm{ref}}\right)}^2} $$

(1)

$$ \mathrm{RMS}{\mathrm{E}}_{\mathrm{tw}}\%=\mathrm{RMS}{\mathrm{E}}_{\mathrm{tw}}/\mathrm{mean}\left({d}_{h_{\left(1,m\right)}}^{\mathrm{ref}}\right) $$

(2)

$$ {\mathrm{bias}}_{\mathrm{tw}}=\frac{1}{m}{\sum}_{i=1}^m\left({d}_{h_i}^{\mathrm{ext}}-{d}_{h_i}^{\mathrm{ref}}\right) $$

(3)

$$ {\mathrm{bias}}_{\mathrm{tw}}\%={\mathrm{bias}}_{\mathrm{tw}}/\mathrm{mean}\left({d}_{h_{\left(1,m\right)}}^{\mathrm{ref}}\right) $$

(4)

where m is the total number of the stem diameters, i is the index of the stem diameter estimates, h_i is the tree height of the i^th stem diameter, ${d}_{h_i}^{\mathrm{ext}}$ is the extracted stem diameter at tree height h_i, and ${d}_{h_i}^{\mathrm{ref}}$ is the reference stem diameter at tree height h_i. The tree-wise accuracy measures RMSE_tw, RMSE_tw%, bias_tw, and bias_tw% were averaged over all extracted stem curves and the averaged values are denoted by RMSE, RMSE%, bias, and bias%, respectively.

The length of a stem curve model was evaluated using two criteria, namely, the curve length ratio (CLR) and the percentage of height retrieved by curve (PHC). For a tree, CLR equals the ratio between the lengths of the automated modeled and the reference stem curves, and PHC is the ratio between the length of the modeled stem curve and the reference tree height.

Stem detection and the effective area

The results of manual and automated stem detection from the point cloud data are reported in Table 1, and illustrated in Fig. 4. The reference trees were divided into three groups with respect to their DBH, namely DBH between 5 and 10 cm, between 10 and 15 cm, and greater than 15 cm.

Table 1 Accuracy of stem detection in test area

Fig. 4

The Stem detection results in relation with the trajectory of data collection. a The trajectory and the inside-canopy-ULS point cloud. The green square marks the effective area of data collection, which was defined according to distribution of the under-canopy trajectory and the locations of the hover-rotation model of the UAV; b Results of manual stem detection with the spatial distribution of inside-canopy-ULS data at the background. Purple dots represent the reference trees whose stems were manually detected, green dots represent the reference trees that were manually undetected; c Results of the automated stem detection. A reference tree is marked with a blue ‘+’ when outside the effective area, and with a red ‘x’ when inside the effective area. A detected tree is marked with an orange circle and a yellow square when inside and outside the effective area, respectively

The total number of manually detected trees was 130. The total number of automatically detected trees was 103, with 102 matched to the reference trees. The one commission was a small tree (DBH < 5 cm) that was not recorded in the reference data. Considering trees with DBH larger than 5 cm, the correctness of the automated stem detection was 100%. The overall completeness of the manual and automated stem detection was 64.00% and 50.25%, respectively. In the whole test area, the ratio between automated and manual detected stems was 78.46%.

The results in the three tree groups suggested that the accuracy of both manual and automated stem detection had a weak correlation with the stem size. This result revealed that although the above-canopy UAV flight covered the whole test site, the coverage of stems was more related with the under-canopy UAV flight in the current flight and hardware configuration. As illustrated in Fig. 4, the hover-rotation of the under-canopy UAV performed similarly as a single scan in a multi-scan TLS approach. Trees that are located closer to those locations were digitized with denser points.

Considering the distribution of the under-canopy trajectory and the locations of the hover-rotation model of the UAV, a subarea was defined in the test area as illustrated in Fig. 4a. It has a size 70 m × 70 m, where all the under canopy trajectories were included and the locations of hover-rotation operations have at least 10 m distance to the boundary. The subarea was regarded as the effective operation area of the inside canopy UAV in this experiment.

The results of stem detection in the effective area are listed in Table 2. The correctness of the automated stem detection was 100%. The overall completeness of the manual stem detection was 80.00%, and that of the automated stem detection was 96.36%.

Table 2 Accuracy of stem detection in the effective area

As mentioned earlier, in the manual detection, the operator ignored those stems that were regarded as difficult or impossible for the manual DBH estimation. Nevertheless, some of the ignored trees in manual stem detection were detected and modeled by the automated method, which explains the higher completeness of the automated stem detection. These results indicate that within the effective area, close to 100% of the trees were digitized to a degree that their stems can be automatically recognized and modeled. The results suggested that the inside canopy ULS has comparable plot-level stem digitization completeness as the multi-scan TLS, in open and sparse stand conditions.

Stem modeling

The evaluation of stem modeling was focused on the effective area, since trees outside the effective area received insufficient data from the under-canopy flight for the stem modelling. Table 3 lists the evaluation results of the stem location, the tree height, and the DBH estimates. The RMSE of the stem positions from the inside-canopy-ULS data is at a level of 10 cm. Compared to previously reported studies, this error is at a similar level to the error of the above-canopy-ULS, larger than that of the multi-scan TLS, i.e., less than 5 cm, and smaller than that of the MLS, i.e., approx. 50 cm (Liang et al. 2019). The RMSE% of the tree height estimates from the inside canopy ULS is at a level of 3%, with a bias% close to zero, and attribute to the contribution of the above-canopy trajectory.

Table 3 Accuracy of the stem characteristics in the effective area

Two evaluation results of the DBH estimates are reported in Table 3 as including and excluding the erroneous estimates. An erroneous estimate is defined when the residual of the DBH estimates exceeded the reference value by 1/3, as shown in the Eq. 5. The comparison between the general evaluation results including and excluding the erroneous estimates indicates the probability of receiving largely biased DBH estimates, moreover, the influences of those largely biased estimates brought to the evaluation results.

$$ \left|{\mathrm{DBH}}_{\mathrm{estimate}}-{\mathrm{DBH}}_{\mathrm{reference}}\right|/{\mathrm{DBH}}_{\mathrm{reference}}>1/3 $$

(5)

Altogether eight erroneous estimates were found, with one from the group of DBH 5–10 cm, two from the group of DBH 10–15 cm, and five from the group of DBH >  15 cm. Regardless of the stem size, the overall RMSE of DBH estimates is between 7 and 8 cm including the erroneous estimates, and between 2 and 4 cm excluding the erroneous estimates. The bias of DBH estimates is between 3 and 4 cm including the erroneous estimates, and close to zero excluding the erroneous estimates. Such performance is better than that of the above-canopy-ULS (Liang et al. 2019), given the lower RMSE and bias of DBH estimates and the much higher completeness of stem digitization. Overall, the performance of the inside-canopy-ULS-based DBH estimates is at a similar level as that of the MLS.

Detailed evaluations on the stem curves and the total estimated length of the stem curve estimates are reported in Table 4. Similarly to the DBH, results are also reported with and without the erroneous estimate. The identical erroneous estimate definition was applied, assuming that the accuracy of the DBH estimation is an indicator for the accuracy of the stem modeling. The accuracy of both diameter and length of the modeled stem curves suggested that the inside canopy ULS carried out more accurate tree-level stem digitization for small and large trees in a stand. Specifically for the small trees (DBH 5–10 cm), the extracted stem curve can reach more than 95% of the tree height, suggesting the advantage of the moving and close-to-target viewpoints of the under-canopy ULS. Meanwhile, the intermediate trees (DBH 10–15 cm) remained the most challenging group for digitization, which received the highest RMSE and bias of stem-wise stem curve diameters, and the lowest PHC. More analyses on such phenomena are in the Discussion.

Table 4 Accuracy of stem curve modeling in the effective area

Discussion

This work demonstrated the feasibility of inside canopy ULS for forest field observations. The high mobility of the UAV platform is expected to meet increasing demands for high data completeness and data collection efficiency. However, because the UAV platform has a 3D freedom of motion and a 3D freedom of rotation, it also prompted more challenge for the accurate determination of the position and the orientation of the platform, which further propagate more difficulties on the generation of geometrically accurate point cloud. The experiment in this study demonstrated the capacity and efficiency of such an inside canopy ULS in improving the completeness of tree- and plot-level structure digitization with high efficiency.

General performance of the inside canopy ULS

The accuracy of automated stem detection is determined jointly by the data quality and the algorithm performance. In an open, sparse, and young forest, the data quality has a more significant impact on the automated stem detection and modeling results than the method used. The completeness of the automated tree detection indicates the plot-level completeness of trees in the data, and the accuracy of the stem modeling reflects the quality, especially the geometric accuracy, of the digitized structures in the data. The results of automated stem detection in the effective area indicate that the inside canopy ULS is capable of digitizing close to 100% of the stems at the plot-level. This is a comparable performance to that of the multi-scan TLS.

The stem detection rates in the total covered area and inside the effective area had a clear difference, i.e., 50.25% and 96.36%, respectively. Regarding the fact that the under canopy trajectory of the flight was concentrated in the effective area, these results demonstrate the importance of the under-canopy trajectory in improving the completeness of digitized structures in the point cloud data.

The tree height and the stem curve length results suggested that the inside canopy ULS provided high completeness of tree structures at upper canopy parts as well. The overall bias of tree height is 16 cm, and the bias% is 0.91%, suggesting that the incorporated above-canopy trajectory in the inside canopy ULS has improved the completeness of the upper crowns. The CLR and PHC values of the modeled stem curves suggested that the inside canopy ULS is capable of modeling the stems to a length that is equivalent to that of the MLS and the multi-scan TLS systems. Therefore, the inside canopy ULS is proven to be advantageous for improving both the plot- and tree-level completeness of forest data collection with higher time and cost efficiency.

At the current stage, the major challenge of the inside canopy ULS system stems from the geometric accuracy of the point cloud, which has a determining role on the accuracies of derived tree parameters. In this study, the average RMSE% of the stem curve diameters estimates was approximately 40%, suggesting that the geometric accuracy of the inside canopy ULS data has not yet reached the level of that of the TLS systems. The RMSE values of the DBH estimates excluding the erroneous estimates were between 2 and 4 cm regardless of the stem sizes, suggesting that such errors included systematic effects. A more detailed analysis on the geometric accuracy of the inside canopy ULS is presented in the next section.

Geometric errors of the inside canopy ULS

The geometric accuracy of a point cloud is mainly determined by two factors: 1) the positioning accuracy of the points (returns of laser pulses) that is defined by the LS instrument, which typically degrades with the increase of target to scanner distance, and 2) the positioning accuracy of the platform, which is a joint effect of the platform’s locations and orientations. Theoretically, each point in a point cloud has its own geometric accuracy, which means that the geometric errors of points are heterogeneously distributed along the directions of the laser beams and the trajectory. To simplify the expression, the geometric error ε_p of each point in a point cloud can be understood as a function of errors from the laser scanner ε_scan, and the errors of the platform positioning ε_trajectory, namely:

$$ {\varepsilon}_p=f\left({\varepsilon}_{\mathrm{scan}},{\varepsilon}_{\mathrm{trajectory}}\right) $$

(6)

The laser scanner used in this study is developed specifically for the UAV platform, i.e., Riegl miniVUX-1UAV, which represents the high-end hardware quality and high measurement accuracy for light-weight UAVs. The scanner has a ranging accuracy of 15 mm. The footprint is 160 mm × 50 mm at a 100-m distance. It has multiple target capability, recording up to five target echoes per laser shot. The minimum range of the miniVUX scanner is 3 m, and we found that the effective target to scanner range for data collection is between 20 and 30 m in forest conditions of the test site as illustrated in Fig. 5. The footprint size is thus in a range between 4.8 mm × 1.5 mm at 3 m and 32 mm × 10 mm at 20 m from the scanner. When one considers the ranging accuracy and the footprint size, the measurement error is:

$$ {\varepsilon}_{\mathrm{scan}}=\sqrt{{\varepsilon_{\mathrm{range}}}^2+{\varepsilon_{\mathrm{footprint}\_\mathrm{r}1}}^2+{\varepsilon_{\mathrm{footprint}\_\mathrm{r}2}}^2} $$

(7)

where ε_range = 15 mm; ε_{footprint _ r1}∈ [4.8, 32] mm; and ${\varepsilon}_{\mathrm{footprin}{\mathrm{t}}_{\mathrm{r}2}}\in$ [1.5, 10] mm. Therefore, ε_scan ∈ [15.8, 36.7] mm, that is, within the range of 2–4 cm.

Fig. 5

Point clouds of example trees inside and outside the effective area. The left subfigure marks the locations and the labels of example trees. The right subfigures illustrate the point cloud of the example trees, and the colors of points represent different segments of the trajectory with respect to the platform directions. Green represents the points from the above-canopy flight, other colors are from the under-canopy flight; Labels of the example trees in the left and right figures are correspondent. a Is a tree that was well digitized from all directions (scanned three times from different directions); b Is a tree inside the effective area that was regarded as DBH not measurable by the operator, but was detected and modeled by the automated method; c Is a tree at the border of the effective area; and d is a tree located outside the effective area and with a similar size as the tree in a. Points of tree d mainly come from the above-canopy flight

Because the UAV was flying both above and under the canopy, the ε_trajectory is affected by the inconsistency of GNSS signals due to the blockage from the forest canopies. The trajectory positioning accuracy of the under canopy ULS flight was assumed to be at an equivalent level to that of MLS systems, which was between 50 to 80 cm under boreal forest canopies as reported (Kaartinen et al. 2015; Kukko et al. 2017). In this study, the forest stand of the test area was largely open. Despite several short moments with only three tracked satellites, the GNSS connection remained relatively good for the entire time of the flight. Considering the RMSE values of automated DBH estimates was between 2 and 4 cm, which is at a same range of the ε_scan, the ε_trajectory during the under canopy flights was mitigated by the trajectory optimization provided by RiPrecision tool. At least the misalignments between the data from the under canopy trajectories presented limited impact on the DBH estimates.

Results of the stem curve diameters indicated that the geometric accuracy of points decreased along with the increase of the tree height, as illustrated in Fig. 6. The largest RMSE and RMSE% values of the stem curve diameters come from the intermediate (DBH 10–15 cm) trees, which is 8.27 cm (88.54%) whereas 2.83 cm (36.49%) for small trees (DBH 5–10 cm) and 5.23 cm (26.13%) for big trees (DBH >  15 cm). This indicates that the largest point geometric error ε_p in the point cloud was presented at the middle parts of the canopy, where the points from the above- and under-canopy trajectories merge. The first reason could be a larger ε_trajectory brought by larger misalignment between the above- and under-canopy trajectories, mainly due to the change of tracked number of GNSS satellites. The second reason could be a larger ε_scan from both the above- and the under-canopy trajectories at the middle parts of the canopies where there is long scanner-to-target distances from both above- and under-canopy trajectories. Namely, at the middle parts of the canopy, both above- and under-canopy data present larger ε_scan, and the ε_trajectory brought by the misalignment between the above- and under-canopy trajectories is more significant. Such results suggested the need for intermediate trajectories between the above- and the under-canopy flights to enhance the geometric accuracy of the intermediate canopies.

Fig. 6

Quality of stem points of example trees. The mismatch in the point cloud data becomes more significant as the scanner-to-target distance increases, e.g., at the upper canopies and at the border. a, b and c are the same example trees as in Fig. 5. In each subfigure, the left column illustrates the points (blue) of the example tree. The reconstructed stem model (brown) is illustrated together with the original points. The green rectangles on the stem mark the locations of the stem slices illustrated in the right column. The right column illustrates the x-y projection of the stem slices. Colors of points represent different segments of the trajectory. The stem slices are ordered with respect to their locations on the stem; the thickness of each slice is approx. 20 cm; the width of each slice image is equal to 50 cm; the orange circle in the bottom slice marks the location of the tree and the size of the field measured DBH. The DBHs of tree a, b and c are 41.61, 16.55, and 38.95 cm, respectively

Furthermore, there was approximately 12 cm RMSE of the stem locations which is approximately constant with respect to all tree groups, indicating the existence of a general shift of the point cloud in the world coordinate system brought by limited by the overall geo-referencing accuracy of the trajectory. Such results represent a basic situation where the geo-referencing of the trajectories and the calculation of the point cloud were relied only on the GNSS-IMU on board the ULS system, without any other referencing information. It can thus be assumed that with additional measures to enhance the geo-referencing and the trajectory positioning, the outcomes of such inside canopy ULS system could be improved further.

Outlook

The above- and under-canopy ULS operations are two main aspects of a comprehensive concept of autonomous forest inventory (Jaakkola et al. 2017). The idea of this experiment was to solve the occlusion problem in forests by providing efficient seamless above- and under-canopy forest digitization using one single ULS system. More specifically, the above-canopy trajectory carry out a high-quality digitization of the upper crowns, leading to accurate estimates for tree parameters such as the tree height and crown size. Meanwhile, the under-canopy trajectory is expected to carry out detailed and complete digitization of tree stem structures that could reach or even surpass the quality of multi-scan TLS by efficiently densifying data acquisition positions.

In this experiment, the above-canopy and under-canopy ULS were seamlessly integrated into a single operation of inside canopy ULS flight. At the current stage, the UAV flight was manually controlled, which requires preferably planned trajectory prior to the flight. To manually guide the UAV to avoid collision with branches, the operator had to walk into the forest and to follow the UAV during the whole operation. The applicability of current system setting was limited to boreal forest with relatively open stands, which was unlikely adoptable immediately in more complex sub-tropical and tropical forest considering the size and the safety of the system. Nevertheless, results of the experiment have indicated a comparable plot- and tree-level completeness of tree digitization to that of multi-scan TLS with much higher efficiency and higher completeness of upper crowns. The adaptability of seamless integration of above- and under-canopy ULS in complex forest stands and forest types can be expected to be fundamentally improved in next five to ten years, facilitated by the advancement of autonomous drone navigation in forested environment and the minimization of drone platforms and laser scanning sensors.

The geometrical accuracy of the inside canopy ULS point cloud data has not yet reached to the level of multi-scan TLS, mainly because of the limitations in the performance of the used scanner and GNSS-IMU trajectory determination. Nevertheless, accurate platform positioning is a great challenge for all mobile systems, which requires new solutions other than SLAM (Kukko et al. 2017). In general, point clouds from mobile LS systems have not yet reached the accuracy required by practical forest inventories. However, stationary systems such as TLS have limited efficiency of data acquisition considering the required works of multi-scan data collection and consequent data registration. Therefore, automated mobile systems remain as a crucial direction that is worth more efforts for advancement.

Currently, the geometric errors in the point cloud of inside canopy ULS are likely to be mitigated with a better trajectory design, e.g., adding flights at different height levels above the ground, and with the improvements in the optimization tools to use nature objects instead of the planar surfaces that are hardly available in forests. Meanwhile, new generations of scanners are emerging at a rapid pace with improving measurement capabilities, e.g., ranging accuracy and beam precision.

Results in this study suggested that an all-direction ULS operation which flies freely above, in the middle of, and under the forest canopy could be a new option toward a fully autonomous in situ forest inventory. Two main tasks for such an autonomous ULS-based forest in situ inventory include the optimization of the trajectory design and the automatization of the platform operation. In this study, the under-canopy trajectory was designed to simulate an integrated multi-scan TLS, and the UAV platform remained at a relatively stable height of 1.5 m above the ground. However, the actual advantage of the ULS system is the potential of an all-direction trajectory inside forest, which is the key to a complete and detailed digitization of forest structures. The art lies in the design of the trajectory, which takes into account the coverage, the efficiency, the geometric layout of the scanning operation, and the accuracy of the data, as well as the accessibility and safety of the system. The system automation is complex and challenging, as a UAV has six dimensions of freedom in motion, i.e., the translations and rotations with respect to the X, Y, and Z axis. Several recent studies touched the problem of autonomous drone navigation inside the forest, e.g., (Chisholm et al. 2013; Dionisio-Ortega et al. 2018; Maciel-Pearson et al. 2018), and many other studies of drone navigation in other GNSS-denied environments are also inspiring (Prakash et al. 2019; Wang et al. 2019). Nevertheless, a fully automated operation of an inside canopy ULS is still at a preliminary stage. Meanwhile, semi-autonomous approaches can start to serve the purpose of efficient in situ forest inventory in frame of the concept of seamless integration of above- and under- canopy ULS operations.

Conclusion

In this study, an inside canopy ULS operation was carried out by seamlessly integrating the above- and under-canopy trajectories of an ULS system in a single flight operation. We proofed the feasibility of such concept with an experiment in a boreal forest. The data collection lasted for ca. 10 min over an approximately 0.5 ha effective area. The plot-level completeness of individual trees in the effective area was 96.36%. As for the tree-level completeness, the average ratio between the extracted stem curve length and the reference stem curve length was 80.41%, and the average ratio between the extracted stem curve length and the tree height was 64.35%. Such results were comparable with multi-scan TLS data. The average RMSE of estimated tree height was 0.61 m and the plot-level bias of tree height estimates was 0.16 m; this was a comparable performance to that of the above-canopy ULS, and a better performance than that of multi-scan TLS.

The working times required by the multi-scan TLS, which is at least 10 min per scan including the placing the equipment and the data collection. Even with more advanced TLS system such as Riegl VZ400i when the registration of multiple scans could be easier and faster, the time required for the placement of the instruments in the field would remain approximately the same. Moreover, the weakness of the TLS system at the upper canopy parts can hardly be overcome without the support from other systems. Thus, the advantages of seamless integration of above- and under-canopy ULS are significant, in comparison with the multi-scan TLS.

For the trees in the effective area, the overall RMSE of DBH estimates was 3.76 cm (15.23%) and the bias of DBH estimates was 1.28 cm (5.16%). The overall RMSE of the stem curve diameters was 5.68 cm (39.30%) and the bias of the stem curve diameters was 3.43 cm (24.48%). Such results are comparable to MLS systems when no additional measure was applied to enhance the trajectory positioning. Furthermore, it should be reminded that the test site of this experiment consisted of a young stand where the average DBH was 25.41 cm and the average tree height was 17.85 m. Such young stand is typically challenging for accurate stem modeling.

The experiment results suggested that the inside canopy ULS, which seamlessly integrated the above- and under-canopy observations, has the potential to carry out a close-to-multi-scan-TLS-level forest digitization with an efficiency that is at least equivalent to that of MLS, with additional advantage of high completeness at the upper parts of canopies from the above canopy ULS. Thus, it is a solution that combines the advantages of the multi-scan TLS, the MLS, and the above-canopy ULS, which is a promising step toward a fully autonomous in situ forest inventory. Future studies are required to improve the geometric accuracy of the point cloud data, and to automatize the UAV operation.