Detecting the changes of buildings in urban environments is essential. Existing methods that use only nadir images suffer from severe problems of ambiguous features and occlusions between buildings and other regions. Furthermore, buildings in urban environments vary significantly in scale, which leads to performance issues when using single-scale features. To solve these issues, this paper proposes a fused feature pyramid network, which utilizes both color and depth data for the 3D verification of existing buildings 2D footprints from oblique images. First, the color data of oblique images are enriched with the depth information rendered from 3D mesh models. Second, multiscale features are fused in the feature pyramid network to convolve both the color and depth data. Finally, multi-view information from both the nadir and oblique images is used in a robust voting procedure to label changes in existing buildings. Experimental evaluations using both the ISPRS benchmark datasets and Shenzhen datasets reveal that the proposed method outperforms the ResNet and EfficientNet networks by 5% and 2%, respectively, in terms of recall rate and precision. We demonstrate that the proposed method can successfully detect all changed buildings; therefore, only those marked as changed need to be manually checked during the pipeline updating procedure; this significantly reduces the manual quality control requirements. Moreover, ablation studies indicate that using depth data, feature pyramid modules, and multi-view voting strategies can lead to clear and progressive improvements.

检测城市环境中建筑物的变化至关重要。仅使用最低点图像的现有方法遭受建筑物和其他区域之间的模棱两可的特征和遮挡的严重问题。此外，城市环境中的建筑物在规模上差异很大，这在使用单比例尺特征时会导致性能问题。为了解决这些问题，本文提出了一种融合的特征金字塔网络，该网络利用颜色和深度数据对倾斜的图像对现有建筑物的2D足迹进行3D验证。首先，通过3D网格模型渲染的深度信息丰富了倾斜图像的颜色数据。其次，将多尺度特征融合到特征金字塔网络中，以对颜色和深度数据进行卷积。最后，来自天底和倾斜图像的多视图信息将用于强大的投票过程中，以标记现有建筑物中的更改。使用ISPRS基准数据集和深圳数据集进行的实验评估表明，该方法在查全率和查准率方面分别优于ResNet和EfficientNet网络5％和2％。我们证明了所提出的方法可以成功地检测所有已改变的建筑物。因此，在管道更新过程中，仅需要手动检查标记为已更改的那些；这大大降低了手动质量控制的要求。此外，消融研究表明，使用深度数据，特征金字塔模块和多视图投票策略可以带来清晰且逐步的改进。