Most object detection, recognition, and classification are performed using optical imagery. Images are unable to fully represent the real-world due to the limited range of the visible light spectrum reflected light from the surfaces of the objects. In this regard, physical and geometrical information from other data sources would compensate for the limitation of the optical imagery and bring a synergistic effect for training deep learning (DL) models. In this paper, we propose to classify terrain features using convolutional neural network (CNN) based SegNet model by utilizing 3D geospatial data including infrared (IR) orthoimages, digital surface model (DSM), and derived information. The slope, aspect, and shaded relief images (SRIs) were derived from the DSM and were used as training data for the DL model. The experiments were carried out using the Vaihingen and Potsdam dataset provided by the German Society for Photogrammetry, Remote Sensing and Geoinformation (DGPF) through the International Society for Photogrammetry and Remote Sensing (ISPRS). The dataset includes IR orthoimages, DSM, airborne LiDAR data, and label data. The motivation of utilizing 3D data and derived information for training the DL model is that real-world objects are 3D features. The experimental results demonstrate that the proposed approach of utilizing and integrating various informative feature data could improve the performance of the DL for semantic segmentation. In particular, the accuracy of building classification is higher compared with other natural objects because derived information could provide geometric characteristics. Intersection-of-union (IoU) of the buildings for the test data and the new unseen data with combining all derived data were 84.90% and 52.45%, respectively.

中文翻译：

---

使用SegNet结合数字表面模型得出的坡度，纵横比和多向阴影底纹图像进行土地覆盖分类

大多数对象检测，识别和分类都是使用光学图像进行的。由于可见光光谱从物体表面反射的光线范围有限，因此图像无法完全代表真实世界。在这方面，来自其他数据源的物理和几何信息将补偿光学图像的局限性，并为训练深度学习（DL）模型带来协同效应。在本文中，我们建议利用基于卷积神经网络（CNN）的SegNet模型，通过利用3D地理空间数据（包括红外（IR）正射影像，数字表面模型（DSM）和派生信息）对地形特征进行分类。坡度，坡向和阴影起伏图像（SRI）均来自DSM，并用作DL模型的训练数据。实验是使用德国摄影测量，遥感和地理信息学会（DGPF）通过国际摄影测量和遥感学会（ISPRS）提供的Vaihingen和Potsdam数据集进行的。数据集包括红外正射影像，DSM，机载LiDAR数据和标签数据。利用3D数据和派生信息来训练DL模型的动机是，现实对象是3D特征。实验结果表明，所提出的利用和整合各种信息特征数据的方法可以提高DL语义分割的性能。特别是，与其他自然物体相比，建筑物分类的准确性更高，因为导出的信息可以提供几何特征。