Abstract. Airborne and spaceborne platforms are the primary data sources for large-scale forest mapping, but visual interpretation for individual species determination is labour-intensive. Hence, various studies focusing on forests have investigated the benefits of multiple sensors for automated tree species classification. However, transferable deep learning approaches for large-scale applications are still lacking. This gap motivated us to create a novel dataset for tree species classification in Central Europe based on multi-sensor data from aerial, Sentinel-1 and Sentinel-2 imagery. In this paper, we introduce the TreeSatAI Benchmark Archive, which contains labels of 20 European tree species (i.e., 15 tree genera) derived from forest administration data of the federal state of Lower Saxony, Germany. We propose models and guidelines for the application of the latest machine learning techniques for the task of tree species classification with multi-label data. Finally, we provide various benchmark experiments showcasing the information which can be derived from the different sensors including artificial neural networks and tree-based machine learning methods. We found that residual neural networks (ResNet) perform sufficiently well with weighted precision scores up to 79 % only by using the RGB bands of aerial imagery. This result indicates that the spatial content present within the 0.2 m resolution data is very informative for tree species classification. With the incorporation of Sentinel-1 and Sentinel-2 imagery, performance improved marginally. However, the sole use of Sentinel-2 still allows for weighted precision scores of up to 74 % using either multi-layer perceptron (MLP) or Light Gradient Boosting Machine (LightGBM) models. Since the dataset is derived from real-world reference data, it contains high class imbalances. We found that this dataset attribute negatively affects the models' performances for many of the underrepresented classes (i.e., scarce tree species). However, the class-wise precision of the best performing late fusion model still reached values ranging from 54 % (Acer) to 88 % (Pinus). Based on our results, we conclude that deep learning techniques using aerial imagery could considerably support forestry administration in the provision of large-scale tree species maps at a very high resolution to plan for challenges driven by global environmental change. The original dataset used in this paper is shared via Zenodo (https://doi.org/10.5281/zenodo.6598390) [Schulz et al., 2022]. For citation of the dataset, we refer to this article.

中文翻译：

---

TreeSatAI Benchmark Archive：用于遥感树种分类的多传感器、多标签数据集

摘要。机载和太空平台是大规模森林测绘的主要数据来源，但对单个物种确定的视觉解释是劳动密集型的。因此，针对森林的各种研究已经调查了多个传感器对自动树种分类的好处。然而，仍然缺乏用于大规模应用的可转移深度学习方法。这一差距促使我们基于来自航空、Sentinel-1 和 Sentinel-2 图像的多传感器数据，为中欧的树种分类创建一个新的数据集。在本文中，我们介绍了TreeSatAI 基准存档，其中包含来自德国下萨克森联邦州森林管理数据的 20 个欧洲树种（即 15 个树属）的标签。我们提出了将最新​​机器学习技术应用于具有多标签数据的树种分类任务的模型和指南。最后，我们提供了各种基准实验，展示了可以从包括人工神经网络和基于树的机器学习方法在内的不同传感器中获得的信息。我们发现残差神经网络（ResNet) 仅使用航拍图像的 RGB 波段，其加权精度得分高达 79%，表现得非常好。该结果表明，0.2 m 分辨率数据中存在的空间内容对于树种分类非常有用。随着 Sentinel-1 和 Sentinel-2 图像的结合，性能略有提高。但是，仅使用 Sentinel-2 仍允许使用多层感知器 ( MLP ) 或光梯度提升机 ( LightGBM ) 的加权精度得分高达 74%） 楷模。由于数据集来自现实世界的参考数据，因此它包含高级别的不平衡。我们发现这个数据集属性对模型对许多代表性不足的类别（即稀缺树种）的性能产生负面影响。然而，表现最好的后期融合模型的分类精度仍然达到 54% ( Acer ) 到 88% ( Pinus）。根据我们的研究结果，我们得出结论，使用航空影像的深度学习技术可以极大地支持林业管理部门提供高分辨率的大规模树种地图，以应对全球环境变化带来的挑战。本文使用的原始数据集通过 Zenodo (https://doi.org/10.5281/zenodo.6598390) [Schulz et al., 2022] 共享。对于数据集的引用，我们参考这篇文章。