Robust 3D reconstruction of building surfaces from point clouds based on structural and closed constraints
Introduction
The surface reconstruction from 3D point clouds is crucial for several applications (Berger et al., 2017, Tachella et al., 2019, Li et al., 2019), including those related to virtual reality (Bruno et al., 2010), smart cities (Yang and Lee, 2017) and robotics (Rajput et al., 2018). Building surface reconstruction is a fundamental task to achieve large-scale 3D city reconstruction. Notably, the most challenging task is to propose a robust algorithm for 3D surface reconstruction to generate reliable building surfaces from multiple-sources data with different sparsity, varying noise levels, and complex structures. To this end, a large amount of research has focused on building surface reconstruction using various methods, including image-based (Musialski et al., 2012, Nan et al., 2015, Wang et al., 2015), aerial LiDAR-based (Verma et al., 2006, Jarzkabek-Rychard and Borkowski, 2016, Wu et al., 2017) and ground LiDAR-based (Chen and Chen, 2008, Wan and Sharf, 2012). Reconstruction of building surfaces in complex real-world scenes poses specific challenges due to various geometric shapes and occlusions. Consequently, automated reconstruction of 3D building surfaces from point clouds yet remained an open challenge (Ochmann et al., 2019, Berger et al., 2017). Many methods have been proposed, including those based on implicit methods (Hoppe et al., 1992, Hoppe et al., 1994), moving least-squares fitting method (Alexa et al., 2003), Multilevel Partition of Unity (Ohtake et al., 2003), Poisson surface reconstruction method (Kazhdan et al., 2006; Alliez et al., 2007), and Bayesian surface reconstruction methods (Jenke et al., 2006, Diebel et al., 2006). These implicit methods sometimes tend to smooth the data. In order to produce a sharper surface, unlike the l1- or l2- norm in many methods, Li et al. (2018) proposed a new minimization method by using l0 gradient. Unfortunately, most of these approaches do not cope with poor quality point clouds. To overcome this, Ganapathi-Subramanian et al. (2018) used structure-aware shape templates to guide the generation of initial primitives. However, a limitation occurs due to the variety of shape templates, and it becomes a challenge to generate new shapes. Li et al., 2017, Mo et al., 2019b adopted neural network architecture to capture the layered structure of the reconstructed object to achieve finer geometric perception of a complex structure. Moreover, some valuable scene priors were used for building surface reconstruction. For example, it is beneficial to reconstruct the building surface fully by extracting high-level global scene features instead of local details (Yumer and Kara, 2012, Oesau et al., 2014). In most urban scenes, buildings can be conceived as a composition of surface structures and their relationships. As such, graph neural networks that are able to describe relationships on graphs of 3D point clouds has attracted widespread attention. Mo et al. (2019a) presented a hierarchical graph network for concurrently combining both part geometry and inter-part relations during network training. Similarly, Li et al. (2018b) used graph neural networks to represent probabilistic dependencies among graphs nodes and edges, which can capture structure and attributes. These methods have certain guiding significance for the identification of relations between primitives.
This paper, presents a new approach to building surface reconstruction with a focus on the closed constraints when selecting global inter-plane relations of primitives to obtain the watertight surface model of regularized primitives. More precisely, we developed an energy minimization strategy, which taking into account structural closure constraints and the adjacency of candidate planes. The closed constraints are applied to ensure the coincidence of the intersecting edges of the adjacent planar surfaces.
The rest of this paper is organized as follows: Section 2 briefly reviews related studies in surface reconstruction. Section 3 presents the motivation to conduct this work. Section 4 gives specific implementation steps for the proposed algorithm, including how a candidate was generated, as well as the closed and regular arrangements of planes selection. Section 5 presents and discusses the experimental results and compares the performance of our method with that of several existing methods. Section 6 concludes of the paper.
Section snippets
Related works
The development of 3D sensing equipment has facilitated the acquisition of point clouds. Reconstruction of building models from 3D point cloud has become the main research content in computer graphics and photography, being implemented a lot of research in the past few decades. The polygonal surface reconstruction aims to obtain dense and complete surfaces. According to the difference of reconstruction strategies, the existing surface reconstruction methods of buildings from point clouds mainly
Motivation
Structural relationships in real scenes are often biased or even wrong due to the lack of points and the presence of noise. According to the Manhattan hypothesis, the building is mainly composed of plane parts in most urban human-made building scenes. Typical planar relationships between them include parallelism, coplanarity, orthogonality, and symmetry, etc. Therefore, we first speculate on possible relationships and generate candidate sets that are likely to be close to the real model based
Method
The proposed method aims to reconstruct the closure and regular arrangements of the planes from a raw point cloud of the scanned buildings, where the regular arrangement refers to the orderly combination of primitives with a certain structural relationship. The proposed method considers three main stages:
- (1)
Initialization: First, the raw point cloud S is segmented by a region growing algorithm into a set of patches {Sj} according to the normal consistency, where j is an index of each block cloud.
Experimental setup
We implemented our method using C++ under the ubuntu system. The number of candidate sets is controlled by setting a threshold during the candidate set generation stage. The distance threshold in the experiment is based on the point density presented in Table 1.
Data_sets
In the experiment, the data sets include an aerial image-based point cloud and a LiDAR point cloud. The image-based data such as Container, Contain_unit, Tower, and Bungalow are from Jimei University, China. The structure of the Bungalow
Conclusions
In this paper, we introduced a novel building reconstruction algorithm based on closed constraints to obtain watertight and complete building surface model, named Clow_Rapter. The algorithm considers the closed constraints of polygon boundaries to perform surface reconstruction based on regular arrangements of planes. The pipeline of the proposed algorithm includes candidate planes generation from an input point cloud, candidate planes regulation, closure constrain, and surface model
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the National Natural Science Foundation of China under grants of 41971424, 61702251, 61701191 and U1605254, the Key Technical Project of Fujian Province under Grant No. 2017H6015, the Natural Sciences and Engineering Research Council of Canada under a grant of 50503-10284, the Science and Technology Project of Xiamen under Grant No. 3502Z20183032, and the Key Project of Xiamen Southern Oceanographic Center under grants No. 18CZB033HJ11.
References (63)
- et al.
From 3D reconstruction to virtual reality: A complete methodology for digital archaeological exhibition
Journal of Cultural Heritage.
(2010) - et al.
Surface reconstruction from unorganized points with l0 gradient minimization
Computer Vis. Image Unders.
(2018) - et al.
Modelling of buildings from aerial LiDAR point clouds using TINs and label maps
ISPRS J. Photogramm. Remote Sens.
(2019) - et al.
Principal components analysis (PCA)
Comput. Geosci.
(1993) - et al.
Automatic reconstruction of fully volumetric 3D building models from oriented point clouds
ISPRS J. Photogramm. Remote Sens.
(2019) - et al.
Indoor scene reconstruction using feature sensitive primitive extraction and graph-cut
ISPRS J. Photogram. Remote Sens.
(2014) - et al.
A regularized volumetric fusion framework for large-scale 3D reconstruction
ISPRS J. Photogramm. Remote Sens.
(2018) - et al.
Grammar-based 3D facade segmentation and reconstruction
Comput. Graphics
(2012) - et al.
Urban building reconstruction from raw LiDAR point data
Computer-Aided Des.
(2017) - Achlioptas, P., Fan, J., Hawkins, R. X., Goodman, N. D., Guibas, L.J., 2019. ShapeGlot: Learning Language for Shape...