This paper describes a pipeline for automatically generating level of detail (LOD) models (digital twins), specifically LOD2 and LOD3, from free-standing buildings. Our approach combines structure from motion (SfM) with deep-learning-based segmentation techniques. Given multiple-view images of a building, we compute a three-dimensional (3D) planar abstraction (LOD2 model) of its point cloud using SfM techniques. To obtain LOD3 models, we use deep learning to perform semantic segmentation of the openings in the two-dimensional (2D) images. Unlike existing approaches, we do not rely on complex input, pre-defined 3D shapes or manual intervention. To demonstrate the robustness of our method, we show that it can generate 3D building shapes from a collection of building images with no further input. For evaluating reconstructions, we also propose two novel metrics. The first is a Euclidean–distance-based correlation of the 3D building model with the point cloud. The second involves re-projecting 3D model facades onto source photos to determine dice scores with respect to the ground-truth masks. Finally, we make the code, the image datasets, SfM outputs, and digital twins reported in this work publicly available in github.com/eesd-epfl/LOD3_buildings and doi.org/10.5281/zenodo.6651663. With this work we aim to contribute research in applications such as construction management, city planning, and mechanical analysis, among others.

本文描述了一种用于从独立建筑物自动生成细节层次 (LOD) 模型（数字孪生）的管道，特别是 LOD2 和 LOD3。我们的方法将运动结构 (SfM) 与基于深度学习的分割技术相结合。给定建筑物的多视图图像，我们使用 SfM 技术计算其点云的三维 (3D) 平面抽象 (LOD2 模型)。为了获得 LOD3 模型，我们使用深度学习对二维 (2D) 图像中的开口进行语义分割。与现有方法不同，我们不依赖复杂的输入、预定义的 3D 形状或手动干预。为了证明我们方法的稳健性，我们展示了它可以从一组建筑物图像中生成 3D 建筑物形状，而无需进一步输入。为了评估重建，我们还提出了两个新颖的指标。第一个是基于欧几里得距离的 3D 建筑模型与点云的相关性。第二个涉及将 3D 模型外观重新投影到源照片上，以确定相对于真实掩码的骰子分数。最后，我们在 github.com/eesd-epfl/LOD3_buildings 和 doi.org/10.5281/zenodo.6651663 上公开了这项工作中报告的代码、图像数据集、SfM 输出和数字孪生。通过这项工作，我们的目标是为建筑管理、城市规划和机械分析等应用领域的研究做出贡献。最后，我们在 github.com/eesd-epfl/LOD3_buildings 和 doi.org/10.5281/zenodo.6651663 上公开了这项工作中报告的代码、图像数据集、SfM 输出和数字孪生。通过这项工作，我们的目标是为建筑管理、城市规划和机械分析等应用领域的研究做出贡献。最后，我们在 github.com/eesd-epfl/LOD3_buildings 和 doi.org/10.5281/zenodo.6651663 上公开了这项工作中报告的代码、图像数据集、SfM 输出和数字孪生。通过这项工作，我们的目标是为建筑管理、城市规划和机械分析等应用领域的研究做出贡献。