Quality safety monitoring of LED chips using deep learning-based vision inspection methods

Highlights:

• Traditional vision method is difficult for the miniaturized inspection of chips.

• DCNN-based detection methods have significant breakthroughs in field of vision.

• The training cycle of DCNN model can’t meet the real-time requirements of industry.

• Design offline training and online recognizing two deep CNN data conversion streams.

Abstract

The surface quality safety inspection of a light-emitting diodes (LED) chip is an indispensable step in the LED production process. The traditional vision inspection algorithm extracts defect features by artificial feature extraction. This method has become increasingly difficult as chips have become more miniaturized, and it is difficult to meet the quality safety requirements. Recently, machine learning-based vision detection methods, especially the models of deep convolution neural network (CNN), have made significant breakthroughs in this field, and their performances greatly exceed the traditional model based on manual features. In this paper, a parallel deep convolution model parallel spatial pyramid pooling network (PSPP-net) for LED chip surface quality detection is proposed. Specifically, this model aims at utilizing the advantages of the spatial pyramid pooling (SPP-net) model to online and offline extract two groups of depth-based CNN features through offline training and online recognition of two depth-based CNN data conversion streams. Then, the features are mixed and intersected in the GPU to form 1024-dimensional image feature vectors. Finally, softmax regression is adopted for defect classification and recognition. The problem of off-line feature training extraction and on-line defect recognition in surface quality safety inspection of LED chips is solved.

Introduction

Light-emitting diodes (LEDs) are widely applied due to advantages such as long service life and low energy consumption. Reliability, stability, and safety should be considered for LED applications. The surface quality safety inspection of chips before encapsulation is a necessary procedure for LED batch production. Generally, an LED chip includes four types of areas, P pole, N pole, TCL area, and Mesa area, as shown in Fig. 1. Defect detection and classification of each area are very important for LED chip manufacturing.

Typical defect types of LED chips collected by Automatic Optic Inspection (AOI) equipment include the P pole (marked as 1), N pole (marked as 2), TCL defect (marked as 3), Mesa defect (marked as 4), and zero-defect chip (marked as 0) as shown in Table 1.

In most cases, the surface defect inspection still depends on manual work, and this may result in misjudgments and reduce the manufacturing efficiency due to human fatigue. Machine vision technologies have been widely used to release humans from labour [1], [2], [3], [4], [5], [6], [7], [8], [9], [10] and can greatly improve efficiency and reliability. The visual inspection method is becoming popular for surface defect detection and can achieve high efficiency. The traditional processes of visual inspection methods for chips [11], [12], [13], [14] are shown in Fig. 2.

Appearance defect inspection, as an important application field of visual inspection, mainly includes the inspection of product surface pit, scratches, cracks, bubbles, pores, abrasions, surface roughness, finish degree, texture, burr, and other defects influencing surface quality. Due to surface defect diversity, the inspection methods adopted thereby include multiple image division and processing methods, such as filtering [15], gray threshold value [16], edge detection [17], area growth [18], template matching [19], mathematical morphology [20], neural network [21], [22], [23], and spatial clustering [24]. There are two problems with the surface defect inspection system. (1) Defect inspection accuracy: Two key indexes can be adopted to evaluate the inspection system performance. The first index, used for evaluating the system capability for inspecting LED chip surface defects, is called the defect inspection rate. The second index, used for evaluating the appearance defect recognition capability, is called the defect recognition rate, wherein the defect size and position are finally inspected for defect classification to form the appearance quality evaluation for production guidance. (2) The efficiency of defect inspection algorithm: To improve the inspection accuracy and robustness, some complex algorithms are usually adopted. However, the system operation time is accordingly increased. Therefore, it is necessary to increase the algorithm speed from the aspects of computer software or hardware to meet the online demand of the industrialization production process.

To avoid preprocessing and reduce human factor influence, a deep learning model is adopted for the feature extraction as the key part in surface defect recognition. The deep learning algorithm aims at simulating the problem analysis process of the human brain and combining and learning the low-layer data to form more abstract high-layer feature expression (attribute, category, and so on) to improve the accuracy of subsequent recognition and classification [25]. Since it was firstly proposed by Hinton et al. [26] in 2006, the deep learning algorithm has continuously received attention from scholars in various countries due to the good experimental effects obtained in a variety of fields, such as handwriting, voice, and human face recognition.

At present, the deep learning algorithm has been applied to the image recognition in different scenes, such as image classification [27], [28], [29], [30], image retrieval [31], [32], [33], [34] and object detection [35], [36], [37], [38], [39], [40], [41], [42], but the specific application thereof in the defect inspection and recognition of LED chips has rarely been researched.

Firstly, the LED surface defect AOI system has to meet the online inspection requirement. The manufacturer has strict requirements for the minimum defect size of the high-density wafer and a high-resolution camera should be adopted. Thus, the system should have large image data capacity and good transmission performance. As a result, stricter requirements are proposed for the system due to such massive data. Secondly, the inspection system needs to adapt to such adverse influences as noise and vibration at the LED chip manufacturers. Thus, the system should have a good online inspection effect and stable long-term operation performance. Therefore, a deep learning-based LED chip defect inspection framework is proposed in this paper. This algorithm aims at first learning massive defect samples to obtain the mapping relationship between the training samples and the zero-defect templates. Then, the reconstructed image and the defect image are compared to realize the defect inspection of the samples.

The remainder of this paper is organized as follows: The first part is aimed at introducing the defect types of LED chips, the traditional visual inspection method, and its existing problems. The second part introduces the features of the universal deep learning network framework. Then, the design of a parallel spatial pyramid pooling network (PSPP-net) for LED chip defect recognition is described. Next, the results of the experiment carried out by PSPP-net for LED chip defect recognition are discussed. The fifth part summarizes the paper.