Classification of rock fragments produced by tunnel boring machine using convolutional neural networks
Introduction
As an important monitoring item for tunnel constructors, fragments cut by tunnel boring machine (TBM) can directly reflect the fragmentation of rock mass during excavation. The degree of rock fragmentation plays an essential role in controlling and reducing the overall production cost. Rock fragmentation is related to many factors, including rock mass properties, TBM specifications (such as TBM diameter, cutter number, cutter spacing, cutter size, and shape), and TBM performance parameters (such as thrust force, torque, revolution per minute, and penetration rate). Therefore, it is invaluable to find the connection between rock fragmentation and these factors. Previous studies have shown how rock mass [1], cutter head principal parameters [2], and penetration rate per revolution [3] influence rock fragmentation, and how to identify rock mass rating based on TBM performance parameters and rock fragmentation [4].
Rock crushing during TBM construction is mainly reflected in large rock fragments. The appearance of large rock fragments can indicate the size or shape of muck is nonuniform, reflecting the unreasonable setting of performance parameters. In addition, problems with the conveyor belt mainly occur due to the size or shape nonuniformity of fragments [5]. Therefore, in the excavation process, the monitoring personnel often needed to uninterruptedly observe the fragments falling on the belt conveyor, pick out the large fragments and adjust the excavation parameters in time. For medium-sized fragments that will not damage the belt, it is necessary to adjust tunnel parameters properly. This method is widely used by engineers to arrive at an approximation [6]. However, its long-time operation and failure to collect essential information make the process cumbersome and inefficient, let alone dangerous.
To improve this situation, many studies have been carried out to determine the size distribution of rock fragments. The conventional approach is the sieving analysis in which a group of screens with different mesh sizes are used [7]. Although it is arguably the most accurate technique among others, it is impractical because it is very time-consuming and needs many resources [8]. Moreover, in TBM construction, it is difficult to achieve fast calculation of rock fragmentation on a belt conveyor and timely treatment.
To solve this problem, many researchers have studied the methods based on digital image processing technology. In digital image analysis, the most difficult step to measure the size of rock fragments is image segmentation [9]. Extensive studies have been carried out on this topic. The existing traditional segmentation approaches for rock fragment images could be summarised as (1) thresholding [10,11], (2) edge detection [[12], [13], [14], [15], [16], [17]], (3) region growing or split & merge [[18], [19], [20]], and (4) morphology [8,9,21].
However, due to the characteristics of rock fragments, some drawbacks can be expected when using traditional image segmentation algorithms:
- a.
Nonuniform lighting, shadows, noise (mud, water, or other fine materials), overlapping, and blocking of fragments and the great range in fragment sizes make delineation extremely difficult using simple edge-detection;
- b.
Background and foreground are uneven when the amount of rock and water on the belt conveyor changes for which a simple thresholding algorithm cannot be applied to segment;
- c.
Each rock fragment may possess a textured surface and multiple faces, which often causes over-segmentation while smaller adjacent rock fragments can also be grouped into larger fragments;
- d.
There is a problem with correctly extracting 3-D information from 2-D images. The fragment sizes in the third dimension need to be assumed [12].
Deep learning methods, especially convolutional neural networks (CNN), have dominated the field of computer vision in recent years. The convolutional kernels of CNN can automatically extract deep features instead of relying on manual feature extraction [68]. After LeNet-5 [22,23], many CNN structures have been proposed in which AlexNet [24] has eight layers, VGG [25] has 16-layer and 19-layer models, and in Google Inception [[26], [27], [28], [29]], the inception module was proposed to solve the problems of excessive training parameters, over-fitting, and gradient dispersion caused by very deep networks. Inspired by the success of CNN for image classification tasks, researchers have adapted CNN to semantic segmentation for feature extraction [30,31]. This method realises the ‘pixel-level’ classification of images and provides a new idea for segmentation. However, this method makes ‘pixel-level’ labelling and manual editing necessary to calculate fragment size distribution. These steps greatly increase the complexity of work. Because of the excellent ability of deep CNN to extract high-level features, many studies directly use it as a feature extractor. Then, a fully connected layer or other classifiers, such as a support vector machine, are used to complete the final classification at ‘image-level’ [[32], [33], [34], [35], [36]].
In this work, the deep CNN was used to automatically classify rock fragment images captured by our timing capture camera mounted vertically above the belt conveyor. Recognising the grading range of the maximum fragment volume in the image is required to determine whether manual picking or adjustment of the tunnelling parameters. Fig. 1 summarises our method flow for rock fragments classification. Section 2 summarises the related research in the image-based rock fragment inspection domain. Section 3 presents the acquisition of rock fragment images and the establishment of the dataset; Section 4 demonstrates the architecture of the CNN model and the training details of the proposed algorithm; Section 5 shows classification performance of the CNN model and the details of high-level features extracted from the target dataset; The results are discussed in Section 6; Section 7 concludes the article.
Section snippets
Literature review
Image-based analysis of rock fragmentation is mainly involved when using mechanical excavation or blasting methods in tunnel engineering and the mining industry [37]. In this method, images are captured from the surface of the rock fragments and then manually or by computer, fragmented rock distribution is determined [38].
One technique is to use digital image processing programmes, which have been developed and have made rapid and accurate blast fragmentation distribution assessment possible [39
Geological condition
Excavated materials (muck) produced by TBM consists of three parts: flaky or blocky rock fragments, rock powder, and structural fillings. When hobs crush rock mass, fracture planes are usually formed along weak structural planes, such as joint surfaces and bedding surfaces of the rock mass. Therefore, the composition of excavated materials and particle sizes of fragments are different for different types of surrounding rocks [4].
According to the result of geological exploration, the chosen
Convolution neural network
CNN was proposed by LeCun et al. [22] inspired by natural mechanisms in the 1990s. CNN comprises several convolutional layers alternately connected with a number of pooling layers and can effectively characterise the essential features of the original image. It requires very little pre-processing of the original image.
Still, challenges exist in using CNN, such as determining the number of convolutional layers, the size of the convolutional kernel, and the learning rate of the network, all of
CNN classification performance
Three other representative CNN models and an unmodified version of AlexNet were trained for comparison. Fig. 9 summaries the test accuracy of LeNet-5, modified AlexNet, VGG-16, Inception-v3, and unmodified AlexNet on the rock fragments dataset. The former two models, which have 7 and 8 layers, respectively, were completely trained by our dataset from the very beginning. However, VGG-16 and Inception-v3 model have 16 and 47 layers, respectively, many times deeper than LeNet-5 and AlexNet, so
Practical implications
Based on the tests, there are some findings when CNN applied to the recognition of rock fragments on a conveyor belt:
The results of convolutional feature visualisation show that our method can effectively extract useful features (edges, corner and height characteristics) of rock fragments. This method avoids the defects of over-segmentation and combination caused by the fuzzy edge and complex background of rock fragments when using the traditional segmentation methods.
A suitable model structure
Conclusions and future work
In this paper, CNN models are designed as a feature extractor and classifier for the detection of large rock fragments. The parameters of CNN are optimised through a series of experiments to obtain high accuracy and low loss. The results demonstrate that CNN presents competitive performance. From our viewpoint, three main contributions are included in the current work. Firstly, a dataset of rock fragments has been set up, and a classification scheme with an engineering application value is
Declaration of Competing Interest
The authors declared that they have no conflicts of interest to this work.
We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.
Acknowledgements
The study was supported by the National Key Research and Development Program of China (Grant No. 2018YFB1702500). The authors would like to thank Mr. Yujie Bai, the manager of the construction site of Sinohydro Corporation, No.6 Construction Bureau Co., Ltd., and Mr. Xinyu Li, an electrical engineer from China Railway Construction Heavy Industry Co., Ltd., for providing technical support and assistance for the installation and wiring of the camera.
References (69)
- et al.
Risk analysis using fault-tree analysis (FTA) and analytic hierarchy process (AHP) applicable to shield TBM tunnels
Tunn. Undergr. Space Technol.
(2015) Real-time measurement of the size distribution of rocks on a belt conveyor
IFAC Proceedings
(1988)- et al.
Classification of surface defects on steel sheet using convolutional neural networks
Mater. Technol.
(2017) - et al.
A novel Chebyshev-wavelet-based approach for accurate and fast prediction of milling stability
Precis. Eng.
(2020) - et al.
Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks
Autom. Constr.
(2018) - et al.
Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images
Autom. Constr.
(2018) - et al.
A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks
Autom. Constr.
(2019) - et al.
Determining the fragmented rock size distribution using textural feature extraction of images
Powder Technol.
(2019) - et al.
Statistical estimation of blast fragmentation by applying Stereophotogrammetry to block piles
Int. J. Rock Mech. Min. Sci.
(2014) - et al.
Particle overlap and segregation problems in on-line coarse particle size measurement
Powder Technol.
(1998)
Automated online measurement of limestone particle size distributions using 3D range data
J. Process Control
Estimating size fraction categories of coal particles on conveyor belts using image texture modeling methods
Expert Syst. Appl.
An improved estimation of size distribution from particle profile measurements
Powder Technol.
Analysis of ore particles based on textural pattern recognition
Miner. Eng.
A neural network-based soft sensor for particle size distribution using image analysis
Powder Technol.
Estimation of particle size distribution on an industrial conveyor belt using image analysis and neural networks
Powder Technol.
Automated cement fragment image segmentation and distribution estimation via a holistically-nested convolutional network and morphological analysis
Powder Technol.
Precise cutterhead torque prediction for shield tunneling machines using a novel hybrid deep neural network
Mech. Syst. Signal Process.
Influence of different thicknesses of marble layers on TBM excavation
Chin. J. Rock Mech. Eng.
Case-based reasoning approach for cutterhead principal parameter design of full face rock tunnel boring machine
J. Mech. Eng.
Influence of Penetration on the Rock-Breaking Efficiency of a TBM Cutter
Modern Tunnell. Technol.
Rock mass rating identification based on TBM performance parameters and muck characteristics
Chinese J. Geotech. Eng.
Blast fragmentation analysis using image processing
Int. J. Min. Geo-Eng.
Principles of Rock Fragmentation
Automated estimation of rock fragment distributions using computer vision and its application in mining
IEE Proceedings-Vision, Image and Signal Processing
Rock Image Segmentation Using Watershed with Shape Markers, 2010 IEEE 39th Applied Imagery Pattern Recognition Workshop (AIPR), Washington, DC, USA
Real-Time Image Analysis of Size and Shape Distributions of Rock Fragments, the AusIMM
The development of a PC, image-based, on-line particle-size analyzer
Mining, Metallurgy & Exploration
Analysis of rock fragmentation using digital image processing
J. Geotech. Eng.
Estimation of the size distribution of fragmented rock in ore mining through automatic image processing
Machine vision methods for estimation of size distribution of aggregate transported on conveyor belts
The split system for analyzing the size distribution of fragmented rock
Industrial applications of image analysis-the IPACS system
Cited by (32)
Convolutional neural network-based model for recognizing TBM rock chip gradation
2024, Automation in ConstructionVision-based size distribution analysis of rock fragments using multi-modal deep learning and interactive annotation
2024, Automation in ConstructionGradient optimization method for tunnel resistivity and chargeability joint inversion based on deep learning
2024, Tunnelling and Underground Space TechnologyAutonomous soil vision scanning system for intelligent subgrade compaction
2024, Automation in ConstructionA review of deep leaning in image classification for mineral exploration
2023, Minerals EngineeringA transfer learning-based YOLO network for sewer defect detection in comparison to classic object detection methods
2023, Developments in the Built Environment