A Review on Recent Advances in Vision-based Defect Recognition towards Industrial Intelligence
Introduction
With the advances of industrial technologies, manufacturing is developing towards intelligence, automation and digitization [1]. This improvement not only increases production efficiency, but also brings a series of new challenges, and one of the biggest is product quality control. To guarantee product quality, full-inspection is a development trend in industrial intelligence, which can save unnecessary losses and improve product quality [4]. Recently, with the wide applications of sensors, the data collection of product quality has been addressed successfully [2,3]. This provides strong supports for the realization of full-inspection. However, full-inspection is still unachievable unless some challenges have been solved. And, one of the bottlenecks is how to recognize these defects accurately as well as effectively [5].
Traditionally, defects are usually recognized manually. But the efficiency is too low to satisfy the requirements of industrial intelligence. For example, in a steel workshop, the recognized area only covered about 0.05% of the total product [6]. Furthermore, the recognition results are also unstable because the inspectors are easy to fatigue.
With the development of computer vision, vision-based defect recognition has drawn increasing attention from both the industrial and academic. Vision-based defect recognition employs computer vision techniques to process defect images, and provides a fast, economical and stable manner for defect recognition. Therefore, it has been widely used in many fields, such as steel [7], wood [8], ceramic [9], fabric [10] and architecture [11].
Vision-based defect recognition is to identify if there are defects in the given images. As shown in Fig. 1, this process generally contains data pre-processing, feature extraction and recognition. Data pre-processing is to collect, clear and standardize the defect images. The recognition generally comprises classification, segmentation, detection and matching. Classification recognizes the defect types, segmentation distinguishes the defect and non-defect areas, detection shows the location of the defect, and matching is to find the most similar template. Feature extraction is an essential component in all these recognition tasks. A good feature extraction manner can enhance defect recognition performance. Traditionally, vision-based defect recognition methods can be categorized into statistical methods, structural methods, filter-based methods and model-based methods. In the past decades, several efforts of vision-based defect recognition have been reported. Chin [12] summarized the vision-based defect recognition methods in the 1980s, Newman and Jain [13] presented an overview in the 1990s. Xie [14] summarized advances in the 2000s. Li and Gu [15] reviewed the techniques for the free-form surface. Neogi et al. [6] reviewed the defect recognition methods for steel surface. Kumar [16] and Ngan et al [4] proposed an overview of fabric defect recognition. However, many significant advances have been reported in artificial intelligence, and many newly-emerged techniques, such as deep learning [17,18], have been introduced into vision-based defect recognition. But there are few comprehensive reviews of these newly-developed methods. Therefore, this paper focuses on the recent advances in vision-based defect recognition, and presents a systematical review by expatiating the advantages, disadvantages and application scenarios.
As shown in Fig. 1, according to the different feature extraction manners, this review divides the recent advances into designed-feature based methods and learned-feature based methods. The designed-feature based methods are mainly based on traditional defect recognition methods, such as statistical methods, structural methods, filter-based methods, and model-based methods. The learned-feature based methods are mainly based on deep learning, and they are divided into convolutional neural network-based methods, autoencoder-based methods, and recurrent neural network-based method. This difference is because the feature extraction in deep learning is quite different from the traditional methods. In the traditional methods, features are extracted by the explicitly designed or selected operators. This relies on expert knowledge, and it is usually fast and lightweight, and the recognition results are also more explainable. While the learned-feature based methods can learn the feature automatically. It relies on less knowledge but requires a large dataset, and the training process is also time-consuming. The differences are also presented in Fig. 1. Besides the systematical review, this paper summarizes the performance metrics. Meanwhile, some research trends and challenges are also discussed.
The remainder of this paper is organized as follows. Section 2 reviews the designed-feature based defect recognition methods. Section 3 summarizes learned-feature based defect recognition methods. Section 4 presents some performance metrics for recognition results. Section 5 discusses some future research trends and challenges. Section 6 is the conclusion.
Section snippets
Designed-Feature Based Defect Recognition Methods
The designed-feature based methods use some explicit operators to extract features from defect images, and use simple classifiers for recognition. This extraction relies on expert knowledge, and it is more targeted to the tasks. As shown in Fig. 1, the designed-feature based methods generally contain statistical methods, structural methods, filter-based methods, and model-based methods. The list of these methods is summarized in Table 1, and more detail is discussed below.
Learned-Feature Based Defect Recognition Methods
The learned-feature based defect recognition methods can extract features and useful information automatically, which avoids the explicit feature design. This advantage makes the learned-feature based defect recognition require less prior or expert knowledge. In the past decade, most of the learned-feature based defect recognition methods are based on deep learning (DL) [82]. Thus, this section will mainly summary these DL methods., which are categorized according to the different model
Performance Metrics for Defect Recognition
The performance metrics evaluate the performance of a defect recognition method. In this paper, the performance metrics are summarized from aspects of accuracy and time.
Challenges and Trends in Vision-based Defect Recognition
The goal of vision-based defect recognition is to fuse human knowledge into the model, and to provide a fast and end-to-end manner to recognize the defect type with little human intervention. More importantly, defect recognition should provide some evidence to help production control. However, as shown in Fig. 8, many challenges must be solved before achieving these goals. This section will discuss the challenges and development trends in vision-based defect recognition. The discussion is based
Conclusion
Vision-based defect recognition is an essential technology to guarantee product quality, and plays an important role in industrial intelligence. Since many advanced techniques, such as deep learning, have been introduced into vision-based defect recognition, a comprehensive review is urgently needed to summarize and promote the development in this field. This paper divided the recent advances in vision-based defect recognition into designed-feature based methods and learned-feature based
Declaration of Competing Interest
The authors report no declarations of interest.
Acknowledgements
National Key R&D Program of China under Grant No. 2018AAA0101700, National Natural Science Foundation of China under Grant No. 51711530038, Program for HUST Academic Frontier Youth Team under Grant No. 2017QYTD04.
References (163)
- et al.
Big data analytics for smart factories of the future
CIRP Ann
(2020) - et al.
Automated fabric defect detection—A review
Image Vis Comput
(2011) - et al.
A semi-supervised convolutional neural network-based method for steel surface defect recognition
Robot Comput Integr Manuf
(2020) - et al.
Automated steel bridge coating rust defect recognition method based on color and texture feature
Autom Constr
(2013) Automated visual inspection: 1981 to 1987
(1988)- et al.
A Survey of Automated Visual Inspection
Comput Vis Image Underst
(1995) - et al.
Free-form surface inspection techniques state of the art review
Comput Des
(2004) - et al.
Automated bridge coating defect recognition using adaptive ellipse approach
Autom Constr
(2009) - et al.
Automated recognition of surface defects using digital color image processing
Autom Constr
(2006) - et al.
Machine Vision Based Image Processing Techniques for Surface Finish and Defect Inspection in a Grinding Process
Mater Today Proc
(2018)