Elsevier

Computers in Industry

Volume 133, December 2021, 103545
Computers in Industry

Pixel-level tunnel crack segmentation using a weakly supervised annotation approach

https://doi.org/10.1016/j.compind.2021.103545Get rights and content

Highlights

  • A novel weakly supervised learning network is proposed to annotate the dataset, which saves time and cost.

  • A crack segmentation based on the DeepLabv3+ model is proposed to improve segmentation accuracy.

  • A publicly available benchmark tunnel scene dataset is introduced that contain 95,623 images.

  • An novel metric is introduced to measure the detected cracks, which includes length and mean width.

Abstract

Automatic crack detection plays an essential role in ensuring the safe operation of tunnels, which is also challenging work in reality. In this paper, an innovative framework, which combines the weakly supervised learning methods (WSL) and the fully supervised learning methods (FSL), is presented to detect and segment the cracks in the tunnel images. Firstly, a WSL-based segmentation network Crack-CAM is proposed to annotate the collected data instead of using the traditional manual annotation process. By applying the proposed E-Res2Net101 structure and tuning some hyper-parameters, an FSL-based method named DeepLabv3+ is optimized to enhance the segmentation performance. After the crack segmentation, the risk levels of the detected cracks are judged using a new evaluation metric. In addition, the mean error of the lengths, the mean widths, and the areas are calculated for different types of cracks. A crack dataset in tunnel scenes that contain 3,921,726 sub-images that are cropped from 521 raw images is built to demonstrate the effectiveness of the presented methods. Based on the proposed dataset, the modified DeepLabv3+ achieves the highest MIoU of 0.786 and the best F1 of 0.865. Besides, the proposed framework combining WSL methods (automatic data annotation) and the FSL methods achieved a performance comparable to the framework that is based on manual annotation and the FSL methods, which demonstrates the WSL-based Crack-CAM can label images correctly.

Introduction

In recent years, the number and the mileage of railway tunnels rapidly increased due to the large amount of traffic construction investment. At the same time, the cracks caused by improper construction, materials, and maintenance are also followed, which seriously affects the service function and service life of the tunnel. In a tunnel project, it is very momentous to detect and evaluate the surface cracks of tunnels over time. However, the traditional crack detection methods rely on manual inspections, which wastes a lot of human resources and time. Therefore, an effective crack segmentation system is urgently needed to overcome these shortcomings.

Due to the advantages of high efficiency and convenience, image processing technologies have been used more frequently to detect segmentation defects and cracks (Dang et al., 2018, Dorafshan et al., 2018, Li et al., 2021, Su et al., 2011). However, the tunnel surface is different from the general concrete pavement and building, and there is usually insufficient light intensity, low contrast, complex background texture, and more noise. These confounding factors on the tunnel images often lead to the traditional image processing technology not achieving the desired result. Some methods that are based on deep learning (DL) have recently made significant progress in computer vision-related tasks. The DL-based defect and crack segmentation application is mainly divided into two research methods: fully supervised learning method (FSL) (Liu et al., 2019, Ren et al., 2020, Song et al., 2019) and the weakly supervised learning method (WSL) (Chen et al., 2021, Dong et al., 2020, Zhu and Song, 2020). Compared with the WSL method, the FSL method has a better segmentation effect, but it needs to spend much time with the process of labeling data. In this study, the WSL and the FSL methods are combined to segment the cracks, and the damage degrees of the detected cracks are evaluated using image post-processing.

The main contributions of this study are listed below.

  • A novel network that is based on the WSL method is presented to annotate the dataset, which economizes the time and the cost of human power.

  • A crack segmentation model based on the FSL methods is optimized to enhance the final segmentation accuracy.

  • A crack dataset in tunnel scenes is established to assess the proposed crack inspection system.

  • Finally, a risk assessment metric is introduced to evaluate the damaged condition of the detected cracks by length and mean width.

The rest of the paper is arranged as follows. Section 2 lists some recent studies from three distinct research methods. The detailed information of the established data is described in Section 3. Section 4 shows the proposed crack segmentation framework. In Section 4, some experiments are conducted to demonstrate the proposed methodology, and the advantages and the disadvantages are discussed in Section 5.

Section snippets

Related work

Some researchers have presently developed various defect segmentation applications that are based on image processing methods. For example, morphological segmentation based on edge detection (MSED) was proposed and compared with the opening top-hat operation (OTHO) method. The result suggests the MSED can obtain a better performance than the OTHO (Su et al. 2011). Six distinct methods based on edge detection were trained and tested using the same dataset, and experimental results show they can

Proposed dataset

In this research, the data is acquired using a deep scanner truck with high-resolution night cameras and LED lights. The collected raw images after image stitching have variant resolutions, which range from 12,614*2,922 to 34,473*2,956. To better fit a neural network, a total of 521 tunnel images are cut into 3,921,726 sub-images of 224*224 with a step length of 100. After cutting the images, the obtained sub-images are manually validated, and the detailed information is described in Table 1.

Overview of the proposed framework

In this section, a diagram of the proposed framework is illustrated. As presented in Fig. 2, the proposed crack segmentation and the assessment system mainly contain three phases. Firstly, a model that combines the WSL method and the self-supervised learning (SSL) method is proposed to obtain the class activation map (CAM). Also, the dense Conditional Random Field (dCRF) and the random walk algorithm (RW) are used to generate the annotation images based on the acquired CAM (see more details in

Experimental results

In this research, all the experiments were performed using a Linux machine that is pre-installed with Ubuntu 18.04. It is equipped with four Tesla V100 PCle 32 GB GPUs, an Intel® Xeon® E5-2698 v4 processor, and 256 GB of DDR4 RAM. Section 5.1 illustrates the effectiveness of the proposed automatic pixel-level annotation network based on WSL methods. Then, Section 5.2 explains how the FSL-based segmentation model is optimized and fine-tuned. After that, some state-of-the-art (SOTA) approaches

Conclusion

This paper presents a novel idea of automatic data annotation using the WSL-based method Crack-CAM, which can save a lot of effort and time. Based on the generated labels from Crack-CAM, an FSL-based DeepLabv3+ model is fine-tuned and trained by changing the architecture and the hyper-parameters to enhance the overall performance. A total of 521 tunnel images were collected from four different locations in Korea, and the corresponding GT-mask images were manually made by experts. In order to

CRediT authorship contribution statement

Hanxiang Wang, Yanfen Li: Conceptualization, Methodology, Data curation. L. Minh Dang, Sujin Lee: Visualization, Investigation. Hyeonjoon Moon: Supervision. Yanfen Li: Writing-Reviewing and Editing

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This work was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1A6A1A03038540) and National Research Foundation of Korea (NRF) grant funded by the Korea government, Ministry of Science and ICT (MSIT) (2021R1F1A1046339).

References (33)

  • Jinsong Zhu et al.

    'Weakly supervised network based intelligent identification of cracks in asphalt concrete bridge deck'

    Alexandria Engineering Journal

    (2020)
  • Jiwoon Ahn et al.

    Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    (2018)
  • Yu-Ting Chang et al.

    Weakly-supervised semantic segmentation via sub-category exploration

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    (2020)
  • Guangyong Chen et al.

    Rethinking the usage of batch normalization and dropout in the training of deep neural networks

    arXiv preprint arXiv

    (2019)
  • Haiyong Chen et al.

    Perovskite nanoparticles@N-doped carbon nanofibers as robust and efficient oxygen electrocatalysts for Zn-air batteries

    Journal of colloid and interface science

    (2021)
  • Liang-Chieh Chen et al.

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Proceedings of the European conference on computer vision (ECCV)

    (2018)
  • Cited by (0)

    View full text