Real-manufacturing-oriented big data analysis and data value evaluation with domain knowledge

Abstract

As one of the most popular topics currently, big data has played an important role in both academic research and practical applications. However, in the manufacturing industry, it is difficult to make full use of the research results for production optimization and/or management due to the low quality of real workshop data. Typical quality problems of real workshop data include the information match degree, missing recessive data, and false error identification. The conventional data analysis methods cannot handle most such issues because these methods fail to consider professional insights into and domain knowledge about the data. The main motivation of this paper is to explore methods for analyzing and evaluating big data with domain knowledge. For this purpose, real production data from a semiconductor manufacturing workshop are adopted as the data object. First, a series of data analysis techniques with domain knowledge are developed for diagnosing the imperfections. Then, corresponding data processing techniques with domain knowledge are proposed for solving those data quality problems according to specific flaws in the data. Furthermore, this paper proposes quantitative calculation methods of data value density to determine the extent to which data quality can be improved by the proposed data processing techniques. Case studies are conducted to demonstrate that data analysis and processing techniques with domain knowledge can effectively handle data quality problems of real workshop data in terms of the information match degree, missing recessive data, and false error identification. The work in this paper has the potential to be further extended and applied to other big data applications beyond the manufacturing industry.