Automatic voxel-based 3D indoor reconstruction and room partitioning from triangle meshes
Introduction
Accurate and up-to-date digital 3D building models are gaining in prevalence with the increasing use of Building Information Modeling (BIM) techniques for the support of all stages of building life cycles (Jung and Lee, 2015, Volk et al., 2014). While BIM techniques are by now well-established in the planning and construction phases of new building projects, older stock buildings frequently lack digital representations or only rudimentary or outdated models exist.
Mobile Augmented Reality (AR) devices like the Microsoft HoloLens1 are equipped with various sensors that facilitate tracking and localization within indoor environments. For convincing placement and interaction of virtual content with the physical surrounding, a 3D model needs to be captured on site. The Microsoft HoloLens for instance is equipped with a Time-of-Flight (ToF) range camera providing range images and preprocessed triangle meshes that can be used in custom applications. Recent evaluation studies found these indoor mapping results to have an accuracy in the range of few centimeters (Hübner et al., 2019, Hübner et al., 2020a, Khoshelham et al., 2019).
Mobile AR devices like the Microsoft HoloLens thus represent an efficient and rather low-cost means for the fast and convenient 3D mapping of indoor environments in comparison to e.g. terrestrial or mobile laser scanning systems. Furthermore, they hold great potential for enriching indoor environments with the in situ visualization of all spatial information concerning building management. They can thus be applied to the task of indoor mapping acquiring the input data for the creation of digital building models as well as the in situ presentation of the derived models and their usage for indoor AR applications for instance in the context of facility management.
However, manually constructing building models from indoor mapping data is a laborious and time-consuming task. Hence, we suggest to use these data for the automatic generation of digital models of indoor environments that can serve as basis for augmenting their physical counterpart with location-dependent informative content in an indoor AR setting.
Currently, reconstructing models of built indoor environments from unstructured three-dimensional geometries is a very active field of research (Ma and Liu, 2018, Kang et al., 2020, Lehtola et al., 2020, Pintore et al., 2020). However, current approaches predominantly focus on point clouds acquired by LiDAR sensors or range cameras as input data. To the best of our knowledge, none of the published approaches on indoor reconstruction so far focuses on triangle meshes as provided by state-of-the-art mobile indoor mapping systems like the Microsoft HoloLens or the Matterport system2.
In the case of the aforementioned systems, the triangle meshes are derived from the primary measurements in the form of range images by means of black-box processes which are not accessible by the user. Nevertheless, recent work hints on them having quite favorable properties for classification and interpretation tasks on indoor mapping data. While triangle meshes are a compact representation of indoor mapping geometries in relation to point clouds and thus more favorable in terms of processing time and memory consumption, they still allow for achieving competitive results in classification tasks (Bassier et al., 2020, Weinmann et al., 2020).
In this paper, we present a novel and fully automatic, voxel-based indoor reconstruction approach to derive semantically-enriched and geometrically completed indoor models in voxel representation from triangle meshes. With this approach, voxels are labeled as ‘Ceiling’, ‘Floor’, ‘Wall’, ‘Wall Opening’, ‘Interior Object’ and ‘Empty Interior’. Here, the class ‘Interior Object’ refers to any kind of object within the room that does not belong to the room surfaces. Room surfaces are geometrically completed in case of holes in the input triangle meshes caused by occlusion or incomplete mapping. Furthermore, the derived interior space is partitioned into rooms and connecting transition spaces. Accordingly, the proposed reconstruction approach extends our previous work (Hübner et al., 2020b) by two modules. The first module is integrated into the previous approach and designed to complete the indoor voxel model in areas, where indoor environments exhibit smaller sections with own smaller ceiling surfaces. The second module relies on the improved and refined voxel model and addresses room partitioning, which also accounts for challenging parts of indoor environments such as given by stairwells. Thus, the presented approach does not necessarily require room surfaces (e.g. walls, ceilings or floors) to be planar or aligned with the coordinate axes as given for Manhattan World scenarios. Furthermore, it does not rely on a clear vertical subdivision into distinct floor levels, and it even allows for slanted floors such as ramps or stair flights. It thus can also be applied to highly challenging indoor environments featuring curved room surfaces and complex vertical room layouts. In addition, the presented approach can easily be modified to be applicable for input data in the form of unstructured point clouds.
In summary, our contributions extend (Hübner et al., 2020b) and consist of.
- •
a novel, voxel-based approach to automatically reconstruct three-dimensional indoor environments from triangle meshes including 3D room partitioning that is applicable to challenging indoor environments (e.g. curved room surfaces or complex vertical room layouts extending over multiple storeys),
- •
a method for the automated quantitative evaluation of indoor reconstruction results against ground truth data, and
- •
providing the code of our implementation of our proposed indoor reconstruction method along with the code for automated evaluation in addition to four benchmark datasets acquired with the Microsoft HoloLens (version 1) with manually generated ground truth data to the community (https://github.com/huepat/voxir).
After briefly summarizing related work in Section 2, we explain our voxel-based reconstruction method for deriving voxel models of indoor environments from triangle meshes in Section 3. In addition, we describe the procedure applied for performance evaluation in detail. Subsequently, we present the achieved results in Section 4, and we discuss these results in Section 5 with respect to different aspects. Finally, we provide concluding remarks as well as suggestions for future work in Section 6.
Section snippets
Related work
In the following, we present an overview on related work associated with the topic of indoor reconstruction. First, in Section 2.1, we focus on sensor systems commonly used for indoor mapping. Subsequently, in Section 2.2, we summarize related approaches for the task of automatically reconstructing semantic and geometric models of indoor environments from the acquired data. Finally, in Section 2.3, we address performance evaluation and the related methodology that can be used to evaluate the
Methodology
A novel method for the reconstruction of voxel models of indoor environments from unstructured 3D data with oriented normals is presented in the following. Fig. 1 visually summarizes the proposed approach, while the colors used for semantic voxel classes are detailed in Table 1. The given input data representing indoor environments are voxelized to a three-dimensional voxel grid. In this voxel representation, a model of the indoor environment is reconstructed by assigning voxels to rooms and
Results
In this study, we present quantitative evaluation results for four datasets of triangle meshes of indoor environments acquired via the Microsoft HoloLens (version 1) depicted in Fig. 8. The dataset ‘Office’ as depicted in Fig. 8(a) represents a two-storey office environment comprised of 24 rooms including a stairwell that connects the two levels vertically. The dataset ‘Attic’ is depicted in Fig. 8(b). It represents an attic environment with slanted ceilings comprised of five rooms. Fig. 8(c)
Discussion
The quantitative evaluation results presented in Section 4 for the metrics introduced in Section 3.7 show in general low room mapping errors over the different datasets, resolutions and rotation angles around the up-axis. This implies an overall successful mapping of segmented room entities between ground truth and test data. Higher room mapping errors of up to 25% occur occasionally, e.g. for the dataset ‘Attic’ at 20° rotation around the up-axis in Table 4 or for the dataset ‘Basement’ at 11
Conclusion
In this work, a novel fully-automatic voxel-based approach for the geometric and semantic reconstruction of indoor environments from triangle meshes is presented. First, the input triangle mesh is converted to a voxel representation, where the voxel values are based on the dominant normal directions of the contained triangles. We suggest using a voxel resolution of 5 cm, however this value is freely adjustable as we demonstrate within the scope of the presented evaluation. Ceiling segments are
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (82)
Generalizing the Hough Transform to Detect Arbitrary Shapes
Pattern Recognit.
(1981)- et al.
A Structured Regularization Framework for Spatially Smoothing Semantic Labelings of 3D Point Clouds
ISPRS J. Photogramm. Remote Sens.
(2017) - et al.
Real-time detection of planar regions in unorganized point clouds
Pattern Recognit.
(2015) - et al.
A Review of 3D Reconstruction Techniques in Civil Engineering and their Applications
Adv. Eng. Inf.
(2018) - et al.
Voxelization Algorithms for Geospatial Applications: Computational Methods for Voxelating Spatial Datasets of 3D City Models Containing 3D Surface, Curve and Point Data Models
MethodsX
(2016) - et al.
Automatic Reconstruction of Fully Volumetric 3D Building Models from Point Clouds
ISPRS J. Photogramm. Remote Sens.
(2019) - et al.
Indoor scene reconstruction using feature sensitive primitive extraction and graph-cut
ISPRS J. Photogramm. Remote Sens.
(2014) - et al.
Scanning geometry: influencing factor on the quality of terrestrial laser scanning points
ISPRS J. Photogramm. Remote Sens.
(2011) - et al.
Building Information Modeling (BIM) for Existing Buildings – Literature Review and Future Needs
Aut. Constr.
(2014) - et al.
Semantic line framework-based indoor building modeling using backpacked laser scanning point cloud
ISPRS J. Photogramm. Remote Sens.
(2018)
Semantics-guided reconstruction of indoor navigation elements from 3D colorized points
ISPRS J. Photogramm. Remote Sens.
Voxel-Based Visibility Analysis for Safety Assessment of Urban Environments
ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
Quantifying the Quality of Indoor Maps
Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
Clustering of Wall Geometry from Unstructured Point Clouds Using Conditional Random Fields
Remote Sens.
Topology Reconstruction of BIM Wall Objects From Point Cloud Data
Remote Sens.
Point Cloud vs. Mesh Features for Building Interior Classification
Remote Sens.
Development of a portable high performance mobile mapping system using the robot operating system. ISPRS Ann. Photogramm. Remote Sens. Spat
Inf. Sci.
Scan-to-BIM Output Validation: Towards a Standardized Geometric Quality Assessment of Building Information Models Based on Point Clouds
Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
Assessing the Accuracy and Precision of Imperfect Point Clouds for 3D Indoor Mapping and Modelling. ISPRS Ann. Photogramm. Remote Sens. Spat
Inf. Sci.
The accuracy comparison of three simultaneous localization and mapping (SLAM)-based indoor mapping technologies
Sens.
Semantic Extraction of Permanent Structures for the Reconstruction of Building Interiors from Point Clouds
Sens.
Structural 3D Reconstruction of Indoor Space for 5G Signal Simulation with Mobile Laser Scanning Point Clouds
Remote Sens.
ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans
Spatial Subdivision of Complex Indoor Environments for 3D Indoor Navigation
Int. J. Geogr. Inf. Sci.
From Point Clouds to 3D Isovists in Indoor Environments
Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
Semantic Enrichment of Octree Structured Point Clouds for Multi-Story 3D Pathfinding
Trans. GIS
Automatic Extraction of a Navigation Graph Intended for IndoorGML from an Indoor Point Cloud. ISPRS Ann. Photogramm. Remote Sens. Spat
Inf. Sci.
Rasterization and voxelization of two- and three-dimensional space partitionings
Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
Navigation in Indoor Voxel Models. ISPRS Ann
Photogramm. Remote Sens. Spat. Inf. Sci.
Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark
ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
Evaluation of the Microsoft HoloLens for the Mapping of Indoor Building Environments
In: Dreiländertagung der DGPF, der OVG und der SGPF in Wien, Österreich - Publikationen der DGPF
Automatic extrinsic self-calibration of mobile mapping systems based on geometric 3d features
Remote Sens.
Structured Indoor Modeling
Indoor Reconstruction from Floorplan Images with a Deep Learning Approach
ISPRS Int. J. Geo-Inf.
The Status of BIM Adoption on Six Continents
Int. J. Civ. Struct. Constr. Arch. Eng.
A Review of Techniques for 3D Reconstruction of Indoor Environments
ISPRS Int. J. Geo-Inf.
Design, calibration, and evaluation of a backpack indoor mobile mapping system
Remote Sens.
Cited by (20)
Automated BIM generation for large-scale indoor complex environments based on deep learning
2024, Automation in ConstructionSemantic-aware room-level indoor modeling from point clouds
2024, International Journal of Applied Earth Observation and GeoinformationKnowledge-driven inference for automatic reconstruction of indoor detailed as-built BIMs from laser scanning data
2023, Automation in ConstructionIndoor mapping using low-cost MLS point clouds and architectural skeleton constraints
2023, Automation in ConstructionBIM generation from 3D point clouds by combining 3D deep learning and improved morphological approach
2022, Automation in ConstructionCitation Excerpt :Traditionally, creating a semantically rich 3D indoor model is generally a manual process. The manual construction of as-built BIMs is time-consuming, labor-intensive, tedious, and subjective, and requires skilled workers [12,16,42]. To accelerate data processing and improve the modeling accuracy, many attempts have been made to automate 3D model creation from 2D drawings or 3D point clouds in the GIS and architecture, engineering, and construction domains [1,35,43–45].