Robust 3D reconstruction of building surfaces from point clouds based on structural and closed constraints

https://doi.org/10.1016/j.isprsjprs.2020.09.004Get rights and content

Abstract

The reconstruction of buildings using inhomogeneous and unstructured point clouds is a challenging task for photogrammetry and computer vision research communities. A new approach for 3D building surface modeling, based on closed constraints, is proposed. First, a region growth algorithm is applied to fit the input point clouds by a set of candidate planes. Then, additional candidate planes are generated from the initial planes according to a rigid transformation followed by expanding the original primitive set to the candidate model set through generation rules. Furthermore, an energy function is employed to combine the data fitting errors with the structural constraints at the model selection stage. Finally, the 3D building surface model is generated from the candidate set through energy minimization. More precisely speaking, we adopt the surface optimization scheme that enforces the 3D polygonal surfaces of the building to be consistent with a priori geometric structures. Our approach was assessed using multi-source datasets with different densities, noise levels covering diverse and complex structures. The experimental results demonstrated that the proposed approach achieves better accuracy and robustness than those of several state-of-the-art methods.

Introduction

The surface reconstruction from 3D point clouds is crucial for several applications (Berger et al., 2017, Tachella et al., 2019, Li et al., 2019), including those related to virtual reality (Bruno et al., 2010), smart cities (Yang and Lee, 2017) and robotics (Rajput et al., 2018). Building surface reconstruction is a fundamental task to achieve large-scale 3D city reconstruction. Notably, the most challenging task is to propose a robust algorithm for 3D surface reconstruction to generate reliable building surfaces from multiple-sources data with different sparsity, varying noise levels, and complex structures. To this end, a large amount of research has focused on building surface reconstruction using various methods, including image-based (Musialski et al., 2012, Nan et al., 2015, Wang et al., 2015), aerial LiDAR-based (Verma et al., 2006, Jarzkabek-Rychard and Borkowski, 2016, Wu et al., 2017) and ground LiDAR-based (Chen and Chen, 2008, Wan and Sharf, 2012). Reconstruction of building surfaces in complex real-world scenes poses specific challenges due to various geometric shapes and occlusions. Consequently, automated reconstruction of 3D building surfaces from point clouds yet remained an open challenge (Ochmann et al., 2019, Berger et al., 2017). Many methods have been proposed, including those based on implicit methods (Hoppe et al., 1992, Hoppe et al., 1994), moving least-squares fitting method (Alexa et al., 2003), Multilevel Partition of Unity (Ohtake et al., 2003), Poisson surface reconstruction method (Kazhdan et al., 2006; Alliez et al., 2007), and Bayesian surface reconstruction methods (Jenke et al., 2006, Diebel et al., 2006). These implicit methods sometimes tend to smooth the data. In order to produce a sharper surface, unlike the l1- or l2- norm in many methods, Li et al. (2018) proposed a new minimization method by using l0 gradient. Unfortunately, most of these approaches do not cope with poor quality point clouds. To overcome this, Ganapathi-Subramanian et al. (2018) used structure-aware shape templates to guide the generation of initial primitives. However, a limitation occurs due to the variety of shape templates, and it becomes a challenge to generate new shapes. Li et al., 2017, Mo et al., 2019b adopted neural network architecture to capture the layered structure of the reconstructed object to achieve finer geometric perception of a complex structure. Moreover, some valuable scene priors were used for building surface reconstruction. For example, it is beneficial to reconstruct the building surface fully by extracting high-level global scene features instead of local details (Yumer and Kara, 2012, Oesau et al., 2014). In most urban scenes, buildings can be conceived as a composition of surface structures and their relationships. As such, graph neural networks that are able to describe relationships on graphs of 3D point clouds has attracted widespread attention. Mo et al. (2019a) presented a hierarchical graph network for concurrently combining both part geometry and inter-part relations during network training. Similarly, Li et al. (2018b) used graph neural networks to represent probabilistic dependencies among graphs nodes and edges, which can capture structure and attributes. These methods have certain guiding significance for the identification of relations between primitives.

This paper, presents a new approach to building surface reconstruction with a focus on the closed constraints when selecting global inter-plane relations of primitives to obtain the watertight surface model of regularized primitives. More precisely, we developed an energy minimization strategy, which taking into account structural closure constraints and the adjacency of candidate planes. The closed constraints are applied to ensure the coincidence of the intersecting edges of the adjacent planar surfaces.

The rest of this paper is organized as follows: Section 2 briefly reviews related studies in surface reconstruction. Section 3 presents the motivation to conduct this work. Section 4 gives specific implementation steps for the proposed algorithm, including how a candidate was generated, as well as the closed and regular arrangements of planes selection. Section 5 presents and discusses the experimental results and compares the performance of our method with that of several existing methods. Section 6 concludes of the paper.

Section snippets

Related works

The development of 3D sensing equipment has facilitated the acquisition of point clouds. Reconstruction of building models from 3D point cloud has become the main research content in computer graphics and photography, being implemented a lot of research in the past few decades. The polygonal surface reconstruction aims to obtain dense and complete surfaces. According to the difference of reconstruction strategies, the existing surface reconstruction methods of buildings from point clouds mainly

Motivation

Structural relationships in real scenes are often biased or even wrong due to the lack of points and the presence of noise. According to the Manhattan hypothesis, the building is mainly composed of plane parts in most urban human-made building scenes. Typical planar relationships between them include parallelism, coplanarity, orthogonality, and symmetry, etc. Therefore, we first speculate on possible relationships and generate candidate sets that are likely to be close to the real model based

Method

The proposed method aims to reconstruct the closure and regular arrangements of the planes from a raw point cloud of the scanned buildings, where the regular arrangement refers to the orderly combination of primitives with a certain structural relationship. The proposed method considers three main stages:

  • (1)

    Initialization: First, the raw point cloud S is segmented by a region growing algorithm into a set of patches {Sj} according to the normal consistency, where j is an index of each block cloud.

Experimental setup

We implemented our method using C++ under the ubuntu system. The number of candidate sets is controlled by setting a threshold during the candidate set generation stage. The distance threshold in the experiment is based on the point density presented in Table 1.

Data_sets

In the experiment, the data sets include an aerial image-based point cloud and a LiDAR point cloud. The image-based data such as Container, Contain_unit, Tower, and Bungalow are from Jimei University, China. The structure of the Bungalow

Conclusions

In this paper, we introduced a novel building reconstruction algorithm based on closed constraints to obtain watertight and complete building surface model, named Clow_Rapter. The algorithm considers the closed constraints of polygon boundaries to perform surface reconstruction based on regular arrangements of planes. The pipeline of the proposed algorithm includes candidate planes generation from an input point cloud, candidate planes regulation, closure constrain, and surface model

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under grants of 41971424, 61702251, 61701191 and U1605254, the Key Technical Project of Fujian Province under Grant No. 2017H6015, the Natural Sciences and Engineering Research Council of Canada under a grant of 50503-10284, the Science and Technology Project of Xiamen under Grant No. 3502Z20183032, and the Key Project of Xiamen Southern Oceanographic Center under grants No. 18CZB033HJ11.

References (63)

  • M. Alexa et al.

    Computing and rendering point set surfaces

    IEEE Trans. Vis. Comput. Graph.

    (2003)
  • P. Alliez et al.

    Voronoi-based variational reconstruction of unoriented point sets

  • N. Amenta et al.

    A new Voronoi-based surface reconstruction algorithm

  • M. Arikan et al.

    O-Snap: Optimization-based snapping for modeling architecture

    ACM Trans. Graph.

    (2013)
  • A. Arsalan Soltani et al.

    Synthesizing 3D shapes via modeling multiview depth maps and silhouettes with deep generative networks

  • M. Berger et al.

    A survey of surface reconstruction from point clouds

    Comput. Graph. Forum.

    (2017)
  • D. Chen et al.

    Topologically aware building rooftop reconstruction from airborne laser scanning point clouds

    IEEE Trans. Geosci. Remote Sens.

    (2017)
  • J. Chen et al.

    Architectural modeling from sparsely scanned range data

    Int. J. Comput. Vision

    (2008)
  • Chum, O., Matas, J., 2005. Matching with prosac - progressive sample consensus. In: 2005 IEEE Conference on Computer...
  • J.R. Diebel et al.

    A Bayesian method for probable surface reconstruction and decimation

    ACM Trans. Graph.

    (2006)
  • H. Fan et al.

    A point set generation network for 3D object reconstruction from a single image

  • Y. Furukawa et al.

    Reconstructing building interiors from images

  • R. Gal et al.

    Surface reconstruction using local shape priors

  • V. Ganapathi-Subramanian et al.

    Parsing geometry using structure-aware shape templates

  • T. Groueix et al.

    A papier-mâché approach to learning 3D surface generation

  • P. Guerrero et al.

    PCPNET learning local shape properties from raw point clouds

    Comput. Graph. Forum.

    (2018)
  • R. Hartley et al.

    Multiple view geometry in computer vision

    (2003)
  • H. Hoppe et al.

    Piecewise smooth surface reconstruction

  • H. Hoppe et al.

    Surface reconstruction from unorganized points

    ACM. SIGGRAPH Comput. Graph.

    (1992)
  • H. Isack et al.

    Energy-based geometric multi-model fitting

    Int. J. Comput. Vision

    (2012)
  • M. Jarzkabek-Rychard et al.

    3D building reconstruction from ALS data using unambiguous decomposition into elementary structures

    ISPRS J. Photogramm. Remote Sens.

    (2016)
  • Cited by (0)

    View full text