Forest managers and nature conservationists rely on precise mapping of single trees from remote sensing data for efficient estimation of forest attributes. In recent years, additional quantification of dead wood in particular has garnered interest. However, tree-level approaches utilizing segmented single trees are still limited in accuracy and their application is therefore mostly restricted to research studies. Furthermore, the combined classification of presegmented single trees with respect to tree species and health status is important for practical use but has been insufficiently investigated so far. Therefore, we introduce Silvi-Net, an approach based on convolutional neural networks (CNNs) fusing airborne lidar data and multispectral (MS) images for 3D object classification. First, we segment single 3D trees from the lidar point cloud, render multiple silhouette-like side-view images, and enrich them with calibrated laser echo characteristics. Second, projected outlines of the segmented trees are used to crop and mask the MS orthomosaic and to generate MS image patches for each tree. Third, we independently train two ResNet-18 networks to learn meaningful features from both datasets. This optimization process is based on pretrained CNN weights and recursive retraining of model parameters. Finally, the extracted features are fused for a final classification step based on a standard multi-layer perceptron and majority voting. We analyzed the network’s performance on data captured in two study areas, the Chernobyl Exclusion Zone (ChEZ) and the Bavarian Forest National Park (BFNP). For both study areas, the lidar point density was approximately 55 points/m² and the ground sampling distance values of the true orthophotos were 10 cm (ChEZ) and 20 cm (BFNP). In general, the trained models showed high generalization capacity on independent test data, achieving an overall accuracy (OA) of 96.1% for the classification of pines, birches, alders, and dead trees (ChEZ) - and 91.5% for coniferous, deciduous, snags, and dead trees (BFNP). Interestingly, lidar-based imagery increased the OA by 2.5% (ChEZ) and 5.9% (BFNP) compared to experiments only utilizing MS imagery. Moreover, Silvi-Net also demonstrated superior OA compared to the baseline method PointNet++ by 11.3% (ChEZ) and 2.2% (BFNP). Overall, the effectiveness of our approach was proven using 2D and 3D datasets from two natural forest areas (400–530 trees/ha), acquired with different sensor models, and varying geometric and spectral resolution. Using the technique of transfer learning, Silvi-Net facilitates fast model convergence, even for datasets with a reduced number of samples. Consequently, operators can generate reliable maps that are of major importance in applications such as automated inventory and monitoring projects.

森林管理者和自然保护主义者依靠来自遥感数据的单棵树的精确映射来有效地估计森林属性。近年来，特别是对死木的额外量化引起了人们的兴趣。然而，利用分段单树的树级方法的准确性仍然受到限制，因此它们的应用主要限于研究。此外，就树木种类和健康状况而言，预先分割的单棵树的组合分类对于实际使用很重要，但迄今为止尚未进行充分的研究。因此，我们介绍了Silvi-Net，这是一种基于卷积神经网络（CNN）的方法，将机载激光雷达数据和多光谱（MS）图像进行3D对象分类。首先，我们从激光雷达点云中分割单个3D树，渲染多个类似轮廓的侧视图图像，并通过校准的激光回波特性丰富它们。第二，分割树的投影轮廓用于裁剪和遮盖正畸MS，并为每棵树生成MS图像补丁。第三，我们独立训练两个ResNet-18网络以从两个数据集中学习有意义的功能。该优化过程基于预训练的CNN权重和模型参数的递归再训练。最后，将提取的特征融合在一起，以基于标准的多层感知器和多数表决进行最终分类。我们使用切尔诺贝利禁区（ChEZ）和巴伐利亚森林国家公园（BFNP）这两个研究区域所捕获的数据对网络的性能进行了分析。对于两个研究区域，激光雷达点密度约为55个点/米²真实正射影像的地面采样距离值为10 cm（ChEZ）和20 cm（BFNP）。一般而言，训练有素的模型在独立的测试数据上显示出很高的泛化能力，对松树，桦树，al木和枯树（ChEZ）的分类，总体准确度（OA）为96.1％；对于针叶，落叶，障碍物和枯树（BFNP）。有趣的是，与仅使用MS图像进行的实验相比，基于激光雷达的图像将OA增加了2.5％（ChEZ）和5.9％（BFNP）。此外，与基线方法PointNet ++相比，Silvi-Net还显示出优越的OA，分别为11.3％（ChEZ）和2.2％（BFNP）。总体而言，我们的方法的有效性通过使用两个自然林区（400–530棵树/公顷）的2D和3D数据集得到了证明，这些数据集是通过不同的传感器模型以及不同的几何和光谱分辨率获得的。通过使用转移学习技术，Silvi-Net甚至可以对样本数量减少的数据集进行快速的模型收敛。因此，操作员可以生成可靠的地图，这些地图在诸如自动清单和监视项目之类的应用程序中至关重要。