Building information modeling (BIM) has a semantic scope that encompasses all building systems, e.g. architectural, structural, mechanical, electrical, and plumbing. Automated, comprehensive digital modeling of buildings will require methods for semantic segmentation of images and 3D reconstructions capable of recognizing all building component classes. However, prior building component recognition methods have had limited semantic coverage and are not easily combined or scaled. Here we show that a deep neural network can semantically segment RGB-D (i.e. color and depth) images into 13 building component classes simultaneously despite the use of a small training dataset with only 1490 object instances. For this task, the method achieves an average intersection over union (IoU) of 0.5. The dataset was designed using a common building taxonomy to ensure comprehensive semantic coverage and was collected from a diversity of buildings to ensure intra-class diversity. As a consequence of its semantic scope, it was necessary to perform pre-segmentation and 3D to 2D projection as leverage for dataset annotation. In creating our deep learning pipeline, we found that transfer learning, class balancing, and prevention of overfitting effectively overcame the dataset’s borderline adequate class representation. Our results demonstrate how the semantic coverage of a building component recognition method can be scaled to include a larger diversity of building systems. We anticipate our method to be a starting point for broadening the scope of the semantic segmentation methods involved in digital modeling of buildings.

建筑信息模型（BIM）的语义范围涵盖了所有建筑系统，例如建筑，结构，机械，电气和管道。自动化，全面的建筑物数字化建模将需要用于图像语义分割和能够识别所有建筑物组件类别的3D重建方法。但是，现有的建筑构件识别方法的语义覆盖范围有限，并且不容易组合或缩放。在这里，我们显示了一个深度神经网络，尽管使用了只有1490个对象实例的小型训练数据集，却可以在语义上同时将RGB-D（即颜色和深度）图像分割为13个建筑组件类。对于此任务，该方法可实现0.5的平均联合相交（IoU）。该数据集使用通用的建筑分类法进行设计，以确保全面的语义覆盖，并从多种建筑物中收集数据以确保类内多样性。由于其语义范围的原因，有必要执行预分段和3D到2D投影作为数据集注释的杠杆作用。在创建深度学习管道时，我们发现转移学习，类平衡和防止过度拟合有效地克服了数据集的边界充分的类表示形式。我们的结果证明了建筑构件识别方法的语义覆盖范围如何可以扩展到包括更大范围的建筑系统。我们预计我们的方法将成为扩大建筑物数字建模所涉及的语义分割方法范围的起点。