Elsevier

Journal of Manufacturing Systems

Volume 62, January 2022, Pages 753-766
Journal of Manufacturing Systems

A Review on Recent Advances in Vision-based Defect Recognition towards Industrial Intelligence

https://doi.org/10.1016/j.jmsy.2021.05.008Get rights and content

Highlights

  • Vision-based defect recognition is an essential technology to ensure product quality and realize industrial intelligence

  • Many advances have been proposed, and some newly-emerged techniques, such as deep learning, have been employed in recent years

  • A comprehensive review of vision-based defect recognition is urgently needed

  • A systematical review of recent advances in vision-based defect recognition from a feature perspective is presented.

  • Some challenges and development trends are discussed

Abstract

In modern manufacturing, vision-based defect recognition is an essential technology to guarantee product quality, and it plays an important role in industrial intelligence. With the developments of industrial big data, defect images can be captured by ubiquitous sensors. And, how to realize accuracy recognition has become a research hotspot. In the past several years, many vision-based defect recognition methods have been proposed, and some newly-emerged techniques, such as deep learning, have become increasingly popular and have addressed many challenging problems effectively. Hence, a comprehensive review is urgently needed, and it can promote the development and bring some insights in this area. This paper surveys the recent advances in vision-based defect recognition and presents a systematical review from a feature perspective. This review divides the recent methods into designed-feature based methods and learned-feature based methods, and summarizes the advantages, disadvantages and application scenarios. Furthermore, this paper also summarizes the performance metrics for vision-based defect recognition methods. And some challenges and development trends are also discussed.

Introduction

With the advances of industrial technologies, manufacturing is developing towards intelligence, automation and digitization [1]. This improvement not only increases production efficiency, but also brings a series of new challenges, and one of the biggest is product quality control. To guarantee product quality, full-inspection is a development trend in industrial intelligence, which can save unnecessary losses and improve product quality [4]. Recently, with the wide applications of sensors, the data collection of product quality has been addressed successfully [2,3]. This provides strong supports for the realization of full-inspection. However, full-inspection is still unachievable unless some challenges have been solved. And, one of the bottlenecks is how to recognize these defects accurately as well as effectively [5].

Traditionally, defects are usually recognized manually. But the efficiency is too low to satisfy the requirements of industrial intelligence. For example, in a steel workshop, the recognized area only covered about 0.05% of the total product [6]. Furthermore, the recognition results are also unstable because the inspectors are easy to fatigue.

With the development of computer vision, vision-based defect recognition has drawn increasing attention from both the industrial and academic. Vision-based defect recognition employs computer vision techniques to process defect images, and provides a fast, economical and stable manner for defect recognition. Therefore, it has been widely used in many fields, such as steel [7], wood [8], ceramic [9], fabric [10] and architecture [11].

Vision-based defect recognition is to identify if there are defects in the given images. As shown in Fig. 1, this process generally contains data pre-processing, feature extraction and recognition. Data pre-processing is to collect, clear and standardize the defect images. The recognition generally comprises classification, segmentation, detection and matching. Classification recognizes the defect types, segmentation distinguishes the defect and non-defect areas, detection shows the location of the defect, and matching is to find the most similar template. Feature extraction is an essential component in all these recognition tasks. A good feature extraction manner can enhance defect recognition performance. Traditionally, vision-based defect recognition methods can be categorized into statistical methods, structural methods, filter-based methods and model-based methods. In the past decades, several efforts of vision-based defect recognition have been reported. Chin [12] summarized the vision-based defect recognition methods in the 1980s, Newman and Jain [13] presented an overview in the 1990s. Xie [14] summarized advances in the 2000s. Li and Gu [15] reviewed the techniques for the free-form surface. Neogi et al. [6] reviewed the defect recognition methods for steel surface. Kumar [16] and Ngan et al [4] proposed an overview of fabric defect recognition. However, many significant advances have been reported in artificial intelligence, and many newly-emerged techniques, such as deep learning [17,18], have been introduced into vision-based defect recognition. But there are few comprehensive reviews of these newly-developed methods. Therefore, this paper focuses on the recent advances in vision-based defect recognition, and presents a systematical review by expatiating the advantages, disadvantages and application scenarios.

As shown in Fig. 1, according to the different feature extraction manners, this review divides the recent advances into designed-feature based methods and learned-feature based methods. The designed-feature based methods are mainly based on traditional defect recognition methods, such as statistical methods, structural methods, filter-based methods, and model-based methods. The learned-feature based methods are mainly based on deep learning, and they are divided into convolutional neural network-based methods, autoencoder-based methods, and recurrent neural network-based method. This difference is because the feature extraction in deep learning is quite different from the traditional methods. In the traditional methods, features are extracted by the explicitly designed or selected operators. This relies on expert knowledge, and it is usually fast and lightweight, and the recognition results are also more explainable. While the learned-feature based methods can learn the feature automatically. It relies on less knowledge but requires a large dataset, and the training process is also time-consuming. The differences are also presented in Fig. 1. Besides the systematical review, this paper summarizes the performance metrics. Meanwhile, some research trends and challenges are also discussed.

The remainder of this paper is organized as follows. Section 2 reviews the designed-feature based defect recognition methods. Section 3 summarizes learned-feature based defect recognition methods. Section 4 presents some performance metrics for recognition results. Section 5 discusses some future research trends and challenges. Section 6 is the conclusion.

Section snippets

Designed-Feature Based Defect Recognition Methods

The designed-feature based methods use some explicit operators to extract features from defect images, and use simple classifiers for recognition. This extraction relies on expert knowledge, and it is more targeted to the tasks. As shown in Fig. 1, the designed-feature based methods generally contain statistical methods, structural methods, filter-based methods, and model-based methods. The list of these methods is summarized in Table 1, and more detail is discussed below.

Learned-Feature Based Defect Recognition Methods

The learned-feature based defect recognition methods can extract features and useful information automatically, which avoids the explicit feature design. This advantage makes the learned-feature based defect recognition require less prior or expert knowledge. In the past decade, most of the learned-feature based defect recognition methods are based on deep learning (DL) [82]. Thus, this section will mainly summary these DL methods., which are categorized according to the different model

Performance Metrics for Defect Recognition

The performance metrics evaluate the performance of a defect recognition method. In this paper, the performance metrics are summarized from aspects of accuracy and time.

Challenges and Trends in Vision-based Defect Recognition

The goal of vision-based defect recognition is to fuse human knowledge into the model, and to provide a fast and end-to-end manner to recognize the defect type with little human intervention. More importantly, defect recognition should provide some evidence to help production control. However, as shown in Fig. 8, many challenges must be solved before achieving these goals. This section will discuss the challenges and development trends in vision-based defect recognition. The discussion is based

Conclusion

Vision-based defect recognition is an essential technology to guarantee product quality, and plays an important role in industrial intelligence. Since many advanced techniques, such as deep learning, have been introduced into vision-based defect recognition, a comprehensive review is urgently needed to summarize and promote the development in this field. This paper divided the recent advances in vision-based defect recognition into designed-feature based methods and learned-feature based

Declaration of Competing Interest

The authors report no declarations of interest.

Acknowledgements

National Key R&D Program of China under Grant No. 2018AAA0101700, National Natural Science Foundation of China under Grant No. 51711530038, Program for HUST Academic Frontier Youth Team under Grant No. 2017QYTD04.

References (163)

  • M. Chu et al.

    Steel surface defects recognition based on multi-type statistical features and enhanced twin support vector machine

    Chemom Intell Lab Syst

    (2017)
  • H.-F. Ng

    Automatic thresholding for defect detection

    Pattern Recognit Lett

    (2006)
  • M. Aminzadeh et al.

    Automatic thresholding for defect detection by background histogram mode extents

    J Manuf Syst

    (2015)
  • S.-W. Yang et al.

    Automatic defect recognition of TFT array process using gray level co-occurrence matrix

    Optik (Stuttg)

    (2014)
  • K. Song et al.

    A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects

    Appl Surf Sci

    (2013)
  • M. Guo et al.

    Supervised dictionary learning supported classifier with feature fusion scheme to noninvasively detect TRISO-particle defects

    J Nucl Mater

    (2019)
  • Y. Zhang et al.

    Defect detection for tire laser shearography image using curvelet transform based edge detector

    Opt Laser Technol

    (2013)
  • D.-M. Tsai et al.

    Defect detection of solar cells in electroluminescence images using Fourier image reconstruction

    Sol Energy Mater Sol Cells

    (2012)
  • P.H. Chen et al.

    Fourier-Transform-Based Method for Automated Steel Bridge Coating Defect Recognition

    Procedia Eng

    (2011)
  • L. Tong et al.

    Differential evolution-based optimal Gabor filter model for fabric inspection

    Neurocomputing

    (2016)
  • L. Bissi et al.

    Automated defect detection in uniform and structured fabrics using Gabor filters and PCA

    J Vis Commun Image Represent

    (2013)
  • B. Das et al.

    A combined wavelet packet and Hilbert-Huang transform for defect detection and modelling of weld strength in friction stir welding process

    J Manuf Process

    (2016)
  • C. Yang et al.

    Defect detection in magnetic tile images based on stationary wavelet transform

    NDT E Int

    (2016)
  • J.-W. Wang et al.

    Singular value decomposition combined with wavelet transform for LCD defect detection

    Electron Lett

    (2012)
  • W.-C. Li et al.

    Wavelet-based defect detection in solar wafer images with inhomogeneous texture

    Pattern Recognit

    (2012)
  • G.-H. Hu

    Automated defect detection in textured surfaces using optimal elliptical Gabor filters

    Optik (Stuttg)

    (2015)
  • T. Ojala et al.

    A comparative study of texture measures with classification based on featured distributions

    Pattern Recognit

    (1996)
  • K.Y. Song et al.

    Defect detection in random colour textures

    Image Vis Comput

    (1996)
  • O. Myklebust

    Zero Defect Manufacturing: A Product and Plant Oriented Lifecycle Approach

    Procedia CIRP

    (2013)
  • J. Wang et al.

    Deep learning for smart manufacturing: Methods and applications

    J Manuf Syst

    (2018)
  • W. Chen et al.

    A New Ensemble Approach based on Deep Convolutional Neural Networks for Steel Surface Defect classification

    Procedia CIRP

    (2018)
  • Y. Zhang et al.

    Welding defects detection based on deep learning with multiple optical sensors during disk laser welding of thick plates

    J Manuf Syst

    (2019)
  • R. Miao et al.

    Online defect recognition of narrow overlap weld based on two-stage recognition model combining continuous wavelet transform and convolutional neural network

    Comput Ind

    (2019)
  • Y. Iwahori et al.

    Defect Classification of Electronic Board Using Dense SIFT and CNN

    Procedia Comput Sci

    (2018)
  • J. Wang et al.

    Big data analytics for forecasting cycle time in semiconductor wafer fabrication system

    Int J Prod Res

    (2016)
  • L. Wen et al.

    Convolutional Neural Network With Automatic Learning Rate Scheduler for Fault Classification

    IEEE Trans Instrum Meas

    (2021)
  • N. Neogi et al.

    Review of vision-based steel surface inspection systems

    EURASIP J Image Video Process

    (2014)
  • Q. Luo et al.

    Generalized Completed Local Binary Patterns for Time-Efficient Steel Surface Defect Classification

    IEEE Trans Instrum Meas

    (2019)
  • O. Silven et al.

    Wood inspection with non-supervised clustering

    Mach Vis Appl

    (2003)
  • S. Vasilic et al.

    The Edge Detecting Methods in Ceramic Tiles Defects Detection

  • J. Cao et al.

    Fabric defect inspection using prior knowledge guided least squares regression

    Multimed Tools Appl

    (2017)
  • X. Xie

    A Review of Recent Advances in Surface Defect Detection using Texture analysis Techniques

    ELCVIA Electron Lett Comput Vis Image Anal

    (2008)
  • A. Kumar

    Computer-Vision-Based Fabric Defect Detection: A Survey

    IEEE Trans Ind Electron

    (2008)
  • K. Simonyan et al.

    Very deep convolutional networks for large-scale image recognition

    (2014)
  • G.E. Hinton et al.

    Reducing the Dimensionality of Data with Neural Networks

    Science

    (2006)
  • Y. Xie et al.

    Study on the identification of the wood surface defects based on texture features

    Opt - Int J Light Electron Opt

    (2015)
  • N.T. Deotale et al.

    Fabric Defect Detection Adopting Combined GLCM, Gabor Wavelet Features and Random Decision Forest

    3D Res

    (2019)
  • S. Sadaghiyanfam

    Using gray-level-co-occurrence matrix and wavelet transform for textural fabric defect detection: A comparison study

  • M.A. Shabir et al.

    Tyre Defect Detection Based on GLCM and Gabor Filter

  • J. Kumar et al.

    GLCM and ANN based Approach for Classification of Radiographics Weld Images

  • Cited by (0)

    View full text