A new graph-based semi-supervised method for surface defect classification

Highlights

• The semi-supervised learning is more suitable in surface defect classification.

• The poor class separation causes semi-supervised algorithms to perform poorly.

• A graph-based semi-supervised method is proposed for improving class separation.

• Multiple micrographs replace a large graph to perform graph convolution.

Abstract

Vision-based defect classification is an important technology to control the quality of product in manufacturing system. As it is very hard to obtain enough labeled samples for model training in the real-world production, the semi-supervised learning which learns from both labeled and unlabeled samples is more suitable for this task. However, the intra-class variations and the inter-class similarities of surface defect, named as the poor class separation, may cause the semi-supervised methods to perform poorly with small labeled samples. While graph-based methods, such as graph convolution network (GCN), can solve the problem well. Therefore, this paper proposes a new graph-based semi-supervised method, named as multiple micrographs graph convolutional network (MMGCN), for surface defect classification. Firstly, MMGCN performs graph convolution by constructing multiple micrographs instead of a large graph, and labels unlabeled samples by propagating label information from labeled samples to unlabeled samples in the micrographs to obtain multiple labels. Weighting the labels can obtain the final label, which can solve the limitations of computation complexity and practicality of original GCN. Secondly, MMGCN divides unlabeled dataset into multiple batches and sets an accuracy threshold. When the model accuracy reaches the threshold, the unlabeled datasets are labeled in batches. A famous case has been used to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed MMGCN can achieve better computation complexity and practicality than GCN. And for accuracy, MMGCN can also obtain the best performance and the best class separation in the comparison with other semi-supervised surface defect classification methods.

Introduction

Surface defect is a common problem in industrial production [1], and the causes of which are various [2]. The correct classification of surface defects can judge the failure of the machine to prevent further losses [3]. However, manual check is time-consuming and labor-intensive, so automatic surface inspection (ASI) becomes a research hotspot [4]. The ASI consists of a data collection device and a data classification system [5]. With the rise of Big Data [6], collecting defect samples becomes convenient. Therefore, how to improve the accuracy of the defect classification system becomes more and more important.

With the improvement of computing power and the convenience of data acquisition, more and more deep learning methods have been applied in surface defect classification [7], [8], [9], [10], [11], [12], [13]. Zhou et al. used convolutional neural network (CNN) for classification of surface defect [7]. Ren et al. proposed a transfer method for steel surface defect classification [8]. Most of them were based on the supervised learning. Since labeling defect samples requires specialized knowledge, it is very difficult to obtain massive labeled samples. Therefore, semi-supervised learning, which trains model with both labeled and unlabeled samples, provides a good way to solve this problem. Gao et al. [14] proposed pseudo-label CNN, which used pseudo-labels to train the model for semi-supervised steel surface defect classification. He et al. [15] trained two classifiers based on different learning strategies to use labeled and unlabeled samples for semi-supervised learning. Di et al. [16] trained a convolutional autoencoder as a feature extractor on unlabeled samples to form a new classifier for defect classification. However, some defect classes are difficult to classify due to the intra-class variations and the inter-class similarities of surface defect [17]. To present it intuitively, this situation is called as the poor class separation in this paper. Poor class separation can influence the performance of the semi-supervised methods and training with small labeled samples will further exacerbate this problem [17]. Therefore, how to improve the class separation for semi-supervised methods becomes important.

Graph convolutional network (GCN), a newly developed method and widely used in semi-supervised learning, can solve this problem [18]. GCN can construct a graph where images are nodes and their relationships are edges, and propagate feature information between connected nodes. Therefore, in feature space, the distance of connected nodes is closer and the distance of the disconnected nodes is further. This is why GCN can improve the class separation. Kipf and Welling [19] proposed the concept of GCN and applied it to semi-supervised learning. Li et al. [20] proposed dimension wise separable graph convolution, which can reduce intra-class variance of node features. Li et al. [21] proposed feature-fusing graph neural network to address the intercommunication of image with other images. Li et al. [22] proposed graph-like attention network, which used global features and local information to extract discriminative features. Sun et al. [23] proposed to learn the graph structure to address huge intra-class variability and high inter-class ambiguity. The GCN methods have a good performance for improving class separation, but it is hard to apply GCN in semi-supervised surface defect classification directly because of the computation complexity. For the space complexity, constructing a graph from all samples will occupy too much memories. For the time complexity, the constructed graph is too large to meet the needs of training with large-scale datasets, which will take longer time to train.

To solve the problems, this paper proposes multiple micrographs graph convolutional network (MMGCN) for semi-supervised surface defect classification. The motivation of the proposed method is to apply GCN in semi-supervised surface defect classification to improve the class separation, and some improvements are made to adapt GCN in the task. Firstly, MMGCN embeds images as nodes and their relationships as edges to construct micrographs and performs graph convolution to improve class separation. Secondly, MMGCN constructs multiple micrographs instead of a large graph to reduce occupied memories and training time. Thirdly, to make full use of the unlabeled dataset, MMGCN divides the unlabeled dataset into multiple batches and trains the model to reach a threshold, then labels the unlabeled datasets by batches and adds them to the labeled dataset. The main contributions are summarized as the following three points. Firstly, a new graph-based method, named MMGCN, is proposed to reduce occupied memories and training time by constructing multiple micrographs instead of a large-scale graph. Secondly, the proposed method is applied in semi-supervised surface defect classification to improve class separation of surface defect. Thirdly, the proposed method is applied on a surface defect dataset and verified that it can improve class separation as well as achieve the best performance compared with other semi-supervised surface defect classification methods. In order to evaluate the performance of MMGCN for semi-supervised surface defect classification, the proposed method is tested on a famous benchmark dataset, NEU-CLS. Firstly, the experiment of the influences of micrographs size is performed in different size of micrographs and different number of labeled samples. This part is to find how the size of micrographs affects the performance of MMGCN under different number of labeled samples. Secondly, the effectiveness of improvement of MMGCN is performed. This part is to research the performance of the graph-based method in class separation and the improvement of MMGCN compared with GCN in computation complexity and practicality. Thirdly, the comparison of MMGCN with other semi-supervised deep learning methods is performed. This part is to find whether MMGCN has a better performance in class separation and overall accuracy than other well performed methods.

The rest of this paper is organized as follows. Section 2 illustrates the background research about GCN. Section 3 introduces the structure of the proposed method. Section 4 presents the experimental results and discussion. Section 5 shows the conclusion and future work.