Identifying manufacturing operational conditions by physics-based feature extraction and ensemble clustering

Highlights

• Innovative processing and feature extraction for the thermal images from hot stamping.

• Unsupervised identification of operational conditions in hot stamping process using ensemble clustering analysis.

• Integration of domain knowledge in data analysis to link the cluster labels to process health status.

Abstract

Manufacturing processes usually exhibit mixed operational conditions (OCs) due to changes in process/tool/equipment health status. Undesired OCs are direct causes of out-of-control production and thus need to be identified. Data-driven OC identification has been widely used for recognizing undesired OCs, yet most methods of this kind require labels indicating the OCs in model training. In industrial applications, such labels are rarely available due to delay, incompleteness or physical constraints in data collection. A typical case is the thermal images acquired by in-process infrared camera and pyrometer, which contain rich information about process health status but are unlabeled. To facilitate data-driven OC identification with unlabeled thermal images, this study proposes a feature extraction-clustering framework that characterizes the heat-affected zone by its temperature profile and performs ensemble clustering on the extracted features to label the data. Domain knowledge from plant manufacturing is incorporated in the framework to map cluster labels to OCs. Both offline OC recovery and online OC identification are studied. Thermal images from hot stamping in automotive manufacturing are used to demonstrate and validate the proposed method. The feasibility, effectiveness and generality are well justified by the case study results.

Introduction

In manufacturing, the condition of production process is in one of its operational conditions. Operational condition (OC) is defined as the health status of system/tool/equipment [1], [2], [3], [4], [5], [6]. A process may have varying OCs due to scheduled/unexpected changes in tool conditions, system configurations, production environments, etc. Correctly and timely identifying the OC of a process enables prevention and quick maintenance reactions to tool wear, machine faults and process degradations.

Traditionally, OC identification can be achieved by model-based approaches [7]. Methods of this kind target reliability and safety issues in industrial processes and build systems for fault diagnosis. Yet, the complicated model design and restrictive mathematical assumptions have compromised their feasibility in real applications. As a contrast, data-driven methods for OC identification are favorable alternatives with the potential of real-time detection of production issues. They handle complicated data types efficiently and generate readily interpretable and comparable results. In today's advanced manufacturing, a huge volume of data can be generated by smart sensors continuously. The resulted data-rich environment lays the foundation for data-driven OC identification.

However, most data-driven OC identification methods are supervised, requiring prior knowledge about the OCs. “Labels” indicating the OCs of the system are needed for model training and validation. Popular data-driven OC identification approaches include control chart, classification analysis, Principal Component Analysis (PCA), Partial Least Squares (PLS), deep learning [8], etc. They all require prior knowledge about OCs for method initialization and implementation.

Unfortunately, in most manufacturing setting, labels about OCs are not readily available at the time of production. For example, a retrospective analysis of maintenance records may reveal process degradation leading up to the failure, but the degradation may not be directly translated to OC labels, leaving the in-line collected data as unlabeled. It's also possible to have partially labeled data. In the hot stamping case, periodic inspection for die wear/equipment damage provides labels for thermal images leading up to the inspection date. Yet, there are numerous thermal images not matched to any inspection dates, resulting in many unlabeled thermal images. Furthermore, production data can be unlabeled due to physical constraints. In laser-based additive manufacturing (LBAM), for example, out-of-control OCs are reflected in part porosity, which is a microstructural issue that cannot be measured precisely with current technology [9]. Therefore, accurate, creditable labels for thermal images from LBAM rarely exist. In manufacturing, in-situ thermal images captured by infrared (IR) camera or pyrometer reflect tool wear, equipment damage, and process degradation. This study aims to develop an unsupervised learning approach based on clustering analysis for data-driven OC identification with unlabeled thermal images.

There are two unique challenges in identifying OCs from thermal images. First, thermal images increase the complexity of feature extraction (FE). According to whether or not labels are required, FE methods can be divided into “supervised” and “unsupervised” FE. Supervised FE is applied to labeled data, with the goal to minimize the distance between samples belonging to the same class [10]. Automated FE with deep learning [6] is a typical practice of this category. However, supervised FE is infeasible for the in-situ thermal images in our study, as our data are unlabeled, and we do not know the ground-truth classes. Unsupervised FE requires no labeled samples. It finds an optimal projection to preserve crucial information [10]. Unsupervised FE has been explored in recent literature about thermal images-based process monitoring. Sun et al. [11] and Li et al. [12] formed features by combining pixels at certain locations in thermal image; Grasso et al. [13] extracted Region of Interest (ROI) by image thresholding; Liu et al. [14] capture image textural information using the gray level co-occurrence matrix. Wang et al. [15] detected the edge lines and performed Hough transform to map edge points to Hough space. Tootoonia et al. [16] converted the image into an unweighted and undirected network graph. Khanzadeh et al. [17] obtained heat-affected zone (HAZ) boundary characteristics with Functional Principal Component (FPCA). It is noted that most of these studies leveraged the physical meanings of HAZs in their FE, instead of applying pure image processing techniques on thermal images as in conventional FE. Such physics-based FE approaches characterize the object from more explainable and intuitive ways. A limitation of the above works, however, is that they emphasized either the object shape or temperature but not both, so the extracted features may not be sufficiently informative for OC identification. It is of strong interest to have an unsupervised FE method that thoroughly characterizes the physical behavior of HAZs in thermal images.

The second challenge is a lack of consistency and robustness when using clustering analysis for OC identification. The major branches of clustering analysis include centroid-based approach (e.g., $K$-means), connectivity-based approach (e.g., hierarchical clustering), density-based approach (e.g., DBSCAN, OPTICS), condensation-based approach (e.g., BIRCH), grid-based approach (e.g., WaveCluster), model-based approach (e.g., K clusters), and randomized search approach (e.g., CLARANS) [18], [19], [20]. Each branch of methods targets one critical aspect. However, none of them are robust across all kinds of data. Depending on a single clustering analysis for OC identification can be biased. To improve consistency and robustness, ensemble clustering (EC) is promoted in this study, which combines multiple clustering models/results to produce more consistent and robust analysis [21]. The major steps in EC include base (individual clustering) selection and ensemble with consensus function [22]. There have been numerous works on EC, e.g., robust clustering ensemble (RCESCC) [23], multiple $K$-means clustering [24], ultra-scalable ensemble clustering (U-SENC) [25], self-paced clustering ensemble (SPCE) [26], cluster-level weighting [22], etc. However, these works mostly relies on certain knowledge of the ground truth, e.g., a handful of labeled instances, for base selection or consensus function, thus are infeasible when no information about the ground truth is available. Besides, they lack a domain-knowledge-guided mapping mechanism from the EC results to OCs. There is a research gap for a practical, unsupervised method for discovering OCs from the HAZ features.

Targeting the above challenges, this study aims to develop an unsupervised learning framework that extracts informative features from thermal images, assigns labels using EC and associate cluster labels to OCs based on empirical evidence from the manufacturing plant. In the proposed framework, two most practically meaningful perspectives, part temperature and shape, are consistently considered to ensure the correspondence between statistically formed clusters and the OCs identified. According to physical properties, FE characterizes the temperature magnitude and shape of a part's HAZ. EC is adopted to integrate individual clustering results for comprehensive, informative analysis. Specifically, $K$-means [20], Ordering Points to Identify Cluster Structure (OPTICS) [27] and HC with complete linkage [20] are respectively applied on temperature profile and shape features using varying model parameters, generating a large group of clustering outcomes. These results are fused with our proposed EC approach to create ensemble clusters. Physics of the part such as the heat allocation and distribution is leveraged to map the ensemble clusters to OCs and then further categorize the OCs into “normal”, “warning”, and “abnormal” working conditions. The three stages, i.e., from features to ensemble clusters, from ensemble clusters to OCs, and from OCs to working conditions, constitute the proposed framework for OC identification.

The remainder of the paper is organized as follows. Section 2 will introduce the data to be used in case study. Section 3 will elaborate the proposed framework with technical details, followed by a case study in Section 4 to demonstrate the generality and effectiveness of the proposed framework. Section 5 will end the paper with conclusion and future directions.