Point clouds and models with semantic information facilitate various indoor automation, ranging from indoor robotics to emergency responses. Studies are currently being conducted on semantic labeling and modeling based on offline mapped point clouds, in which, the performance is strongly limited by the mapping process. To address this issue, we propose a framework to cooperatively perform the three tasks of semantic labeling, mapping, and 3D modeling of point clouds. First, our framework uses a deep-learning-assisted method to perform frame-level point cloud semantic labeling. Subsequently, point cloud frames with semantic labels are used to extract the structural planes of buildings, followed by the generation of line structures from the planes. Then, these frames are used to estimate the initial poses of a 3D sensor for data collection. In the subsequent pose optimization process, the initial poses are optimized under the constraints of the structural planes. Finally, the optimized poses are used to integrate semantic frames and line structures to generate a point cloud map and 3D line model of buildings. The experimental results show that the proposed method achieves better results than the state-of-the-art methods that separately perform one of the two tasks.

具有语义信息的点云和模型促进了各种室内自动化，从室内机器人到应急响应。目前正在研究基于离线映射点云的语义标记和建模，其中性能受到映射过程的强烈限制。为了解决这个问题，我们提出了一个框架来协同执行点云的语义标记、映射和 3D 建模这三个任务。首先，我们的框架使用深度学习辅助方法来执行帧级点云语义标记。随后，点云框架与语义标签用于提取建筑物的结构平面，然后从平面生成线结构。然后，这些帧用于估计 3D 传感器的初始姿势以进行数据收集。在后续的位姿优化过程中，初始位姿在结构面的约束下进行优化。最后，优化后的姿态用于整合语义框架和线结构以生成点云图和建筑物的 3D 线模型。实验结果表明，所提出的方法比分别执行两项任务之一的最新方法取得了更好的结果。