Building information is an essential part of geographic information system (GIS) applications in urban planning and management. However, it changes rapidly with economic growth. Unmanned aerial vehicles (UAV)-based photogrammetry works well in this situation with its advantages of quick and high-resolution data updating. In this paper, in order to improve building extraction accuracy in complex areas where buildings are characterized by various patterns, complex structures, and unique styles, we present a framework which applies deep learning (DL) semantic segmentation to UAV images with digital surface model (DSM) and visible-band difference vegetation index (VDVI). The results show that extraction accuracy improves. The combination of red, green, blue (RGB) and VDVI bands (RGBVI) can effectively distinguish the building area and vegetation. The application of RGB with DSM bands (RGBD) helps separate buildings from ground objects. The combination of RGB, DSM, and VDVI bands (RGBDVI) can identify small buildings which are usually not high and covered partly by tree branches. The proposed method is further applied to an open standard dataset to evaluate its robustness and results indicate an increased overall accuracy from RGB only (93%) to RGBD (97%).

建筑信息是地理信息系统（GIS）在城市规划和管理中的重要组成部分。但是，它随着经济增长而迅速变化。在这种情况下，基于无人机（UAV）的摄影测量技术可以快速且高分辨率地更新数据，因此在这种情况下效果很好。在本文中，为了提高建筑物具有各种样式，复杂结构和独特风格的复杂区域中建筑物的提取精度，我们提出了一个框架，该框架将深度学习（DL）语义分割应用于具有数字表面模型的UAV图像（ DSM）和可见带差异植被指数（VDVI）。结果表明提取精度提高。红色，绿色，蓝色（RGB）和VDVI波段（RGBVI）的组合可以有效地区分建筑物区域和植被。带有DSM波段（RGBD）的RGB的应用有助于将建筑物与地面物体分开。RGB，DSM和VDVI波段（RGBDVI）的组合可以识别通常不高且部分被树枝覆盖的小型建筑物。所提出的方法进一步应用于开放标准数据集以评估其鲁棒性，结果表明总体准确性从仅RGB（93％）提高到RGBD（97％）。