Deep learning based method of longitudinal dislocation detection for metro shield tunnel segment

https://doi.org/10.1016/j.tust.2021.103949Get rights and content

Highlights

  • Tunnel depth images are generated from point clouds for segment joints labeling.

  • Two deep CNNs are designed to label segment joints accurately and completely.

  • RANSAC algorithm improves accuracy and reliability of dislocation value calculation.

Abstract

This paper presents a longitudinal dislocation detection method using an accurate tunnel segment joint labeling algorithm featured by deep CNNs (Convolutional Neural Networks). This method is proposed to be four steps. First, a mobile scanning system is used to acquire 3D point clouds of metro shield tunnels. Then, we use cylinder projection to generate tunnel surface depth images from 3D point clouds for segment joint labeling. Subsequently, two deep CNNs are designed to accurately label the segment joints on the depth images. The first CNN can roughly locate the segment joint positions, and the second precisely label the segment joints. Based on the labeled segment joints, two point data sets are obtained on both sides of each segment joint. By using the RANSAC algorithm, the two point sets can fit into two planes, the equation of which is then calculated to generate the dislocation value of the tunnel segment. Experiment results show that this method can label segment joints integrally and accurately without being affected by nearby tunnel equipment. Compared with traditional image edge detection algorithms (Canny and Sobel with Hough Transform), the CNNs are more powerful in labeling segment joints. When the distance measuring accuracy of scanner is 1.2 mm + 10 ppm, the internal and external accuracy of our detection method are evaluated to be 0.4 mm and 0.9 mm respectively. Compared with the scanning line method, the external accuracy of our method is higher and more reliable when there is tunnel equipment around segment joints.

Introduction

In metro shield tunnels, the longitudinal segment dislocation refers to the dislocation between two adjacent segments belonging to two different rings, which is always caused by uneven environmental pressure on the segments. Relative radial displacement would occur when the pressure difference between the two segments exceeds the bearing capacity of fixing bolts. This relative radial displacement subsequently causes the two segments in two separate cambered surfaces, as shown in Fig. 1.

Segment dislocation reflects the current state of the tunnel and it is a type of damage and defect in metro shield tunnels. Manual inspection has been the most widely used method for metro shield tunnel segments dislocation detection, mainly because this method is easy to implement with conventional measuring tools. It is difficult for field inspectors to detect tunnel segment dislocation timely and efficiently. Moreover, manual inspection may neglect some subtle tunnel deformation. If left undetected, the tunnel is more vulnerable to further damages, such as crack and leakage. In extreme cases, the deformations might jeopardize running safety and cause life casualty or property loss.

With automatic detection and information technologies up to date, limitations of the manual inspection approach could be resolved. Take the 3D laser scanning technology as an example. With a high sampling rate, 3D laser scanning can obtain high-density and high-precision point cloud data of a target surface (Yang et al., 2013). Point cloud processing algorithms have been intensively studied in the past few years, including point cloud registration (Luo and Wang, 2018), segmentation (Liu et al., 2019), classification (Gao et al., 2018) and so on. More specifically, 3D point clouds have been successfully applied to metro tunnel surveying applications, such as deformation monitoring (He and Yang, 2014), water leakage spots detection (Tan et al., 2016), cross-section surveying (Cao et al., 2014) and so on.

In recent years, mobile metro tunnel scanning systems using 3D laser scanning and point cloud algorithms are gradually applied in some Chinese cities for metro tunnel detection. Being a pioneer in this area, Amberg company’s (Amberg company website) product GRP 5000 (GRP 5000 system specification) is widely used in many detection work during the metro tunnels operation period. In the field data collection, GRP 5000 can dynamically obtain high-precision 3D point clouds of a metro tunnel at the speed of 3 ~ 5 km/h. In indoor longitudinal dislocation detection, Amberg’s processing software can generate inner surface images of metro tunnels and locate the segment joint position on these images. And then, Amberg uses a method which is very similar to the scanning line (SL) method (elaborated in Section 3.4) to calculate the dislocation values of different places on segment joint. Amberg’s detection process is automatic with very little human assistance. However, this method is designed only for point clouds obtained by mobile rather than stationary scanning system. Besides, their dislocation value result might be wrong when extra linear tunnel equipment, often electric wire brackets, appear around the segment joints.

Major limitations of current manual and automatic methods for tunnel segment dislocation detections can be summarized as follows:

  • 1.

    The manual inspection method is rather inefficient, and it is hard for operators to measure the dislocation of some segments like those on the top area of the tunnels.

  • 2.

    The application of existing automated methods like Amberg’s is limited and with defects, as they mainly designed for the mobile scanning system.

In the past decade, a powerful machine learning technique named deep learning has dramatically developed, and has been widely used in many fields including image processing and natural language processing. Through building multi-layer neural networks, deep learning algorithms can extract high-dimensional features for complex classification or regression applications and many others, for example, damage and defect detection.

Some researchers have done studies about damage and defect detection by using deep learning method, and has achieved amazing results. Cha et al. (2017) proposed a vision-based method using a deep architecture of convolutional neural networks (CNNs) for detecting concrete cracks. This method used a sliding window of 256 × 256 pixel resolutions to scan an image and build a classifier through training CNNs for judging whether there are cracks in each image cut by the sliding window. This method can find concrete cracks in realistic situations with about 98% classification accuracy. Beckman et al. (2019) proposed a deep learning-based automatic volumetric damage quantification method by using depth camera data. This method used a deep faster region-based convolutional neural network (Faster R-CNN) to detect concrete spalling damage and accurately calculated the volume of the concrete spalling through projecting the detected concrete spalling to the depth steam. This method shows a high average precision in detecting the concrete spalling and a low mean precision error in volume quantifications. In order to provide quasi real-time simultaneous detection of multiple types of damages, Cha et al. (2018) proposed a deep learning-based structural visual inspection method. By training a modified Faster R-CNN, this paper achieved 90.6%, 83.4%, 82.1%, 98.1%, and 84.7% average precision ratings for the five damage types respectively with high speed. Huang et al. (2018) proposed a novel image recognition algorithm for semantic segmentation of crack and leakage defects of metro shield tunnel. This method can detect the crack and leakage in pixel level by using hierarchies of features extracted by fully convolutional network (FCN). Though deep learning technology has achieved many good results, there have been no researchers engaged in studying the deep-learning based method of longitudinal dislocation detection.

In order to detect the longitudinal dislocation of tunnel segments, this paper proposes a deep learning based method to overcome existing limitations. First, we use a mobile scanning system to obtain point clouds of metro shield tunnels, and then use the cylinder projection method to generate tunnel’s depth images based on point clouds. Subsequently, two deep CNNs are designed to label segment joints on the depth images, which can label segment joints integrally and precisely without being influenced by nearby tunnel equipment. For calculating the dislocation value of a labeled segment joint, this paper obtains two point sets covering small areas on both sides of the segment joint around the calculating position. The RANSAC (Schnabel et al., 2007) algorithm is then used to fit two planes based on the two point sets, and the dislocation value is calculate according to the equations of the two planes. This dislocation value method can eliminate the influence of points belonging to the tunnel equipment, thus enhances the calculating reliability. Our contributions are as follows:

  • 1.

    A cylinder projection method is used to generate depth images for segment joints labeling from 3D point clouds.

  • 2.

    Two Deep CNNs are used to label segment joints in order to improve the labeling integrality and accuracy.

  • 3.

    The RANSAC plane fitting algorithm based on the local points is used to enhance the accuracy and reliability of calculating dislocation values.

This paper is organized as follows. We first take a literature review on the existing dislocation of tunnel segments detection in Section 1. In Section 2, our method is introduced in detail, including the 3D Point cloud acquisition of metro shield tunnel, generation of the depth images, the segment joints labeling method based on deep CNNs and the dislocation value calculating method. Experiment results and discussions are presented in Section 3, followed by some conclusion remarks in Section 4.

Section snippets

Materials and methods

The proposed longitudinal dislocation detection method for metro shield tunnel segments is composed by four phases, as shown in Fig. 2. First, a mobile scanning system is used to acquire 3D point clouds of metro shield tunnels. Then, depth images of the tunnel inner surface are generated using the cylinder projection based on 3D point clouds. Subsequently, two deep CNNs are designed to label the segment joints on the depth images. At last, according to each labeled segment joint, two point sets

Dataset generation

In order to test the proposed method, we collected thirteen point clouds from ten different metro shield tunnels by using the mobile scanning car. The max scanning distance is less than 5 m. The total length of the point clouds is about 15 km and the collected point clouds covered straight and curve parts of metro tunnels. Every scanned point contains 3D coordinate (XYZ) and intensity information. We created 1518 depth images with size of 3072 × 2560 as samples by using the method introduced in

Conclusions and future work

In order to detect the longitudinal dislocation for metro shield tunnel segments, this paper proposes an improved method based on 3D point clouds. We first use a mobile scanning system to acquire 3D point clouds of metro shield tunnels. Cylinder projection is then used to generate the tunnel surface depth images from 3D point clouds for labeling segment joints. Subsequently, two deep CNNs are designed to accurately predict the segment joint position on the depth images. The first deep CNN is

CRediT authorship contribution statement

Anbin Yu: Methodology, Writing - original draft, Formal analysis. Wensheng Mei: Conceptualization, Methodology, Supervision, Resources, Project administration. Mulin Han: Methodology.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (30)

  • District, H., Studies, M., 2012. continuously deformation monitoring of subway tunnel based on XXXIX,...
  • Mikhail Edward et al.

    Analysis and adjustment of survey measurements

    (1981)
  • W. Fang et al.

    Intensity correction of terrestrial laser scanning data by estimating laser transmission function

    IEEE Trans. Geosci. Remote Sens.

    (2015)
  • H. Gao et al.

    Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment

    IEEE Trans. Ind. Informatics

    (2018)
  • G.Z. He et al.

    Deformation monitoring for subway tunnels based on TLS

    Adv. Mater. Res.

    (2014)
  • Cited by (0)

    View full text