Discretization of hybrid CPPS data into timed automaton using restricted Boltzmann machines
Introduction
A Cyber–Physical Production System (CPPS) (Monostori, 2014, Lee, 2008, Rajkumar et al., 2010, Evans and Annunziata, 2012, Muhuri et al., 2019) is defined by collaborating computational elements and physical processes. These computational components provide a set of value-discrete and value-continuous signals carrying information about hybrid system behavior.
Analyzing this data by means of machine learning leads to new services such as condition monitoring (Morgan and O’Donnell, 2018, Andonovski et al., 2018), fault detection (Bayar et al., 2015), diagnosis and root cause identification (Pulido et al., 2019, Bunte et al., 2019, Diedrich et al., 2019) and alarm management (Wang et al., 2016, Vasquez Capacho et al., 2017). In many cases the knowledge about system causalities and timings is essential: the changes of discrete signals may lead to a different behavior of continuous signals; event sequences can be classified as incorrect; timing changes hint at upcoming problems; system dependencies can be used to identify the root causes based on alarms; and known system timings outline potential system optimizations. However, the concepts of causalities and timings are closely associated with discrete events, sequences of events and timings between these events—all undefined for the continuous part of the system.
The existing approaches often solve these challenges by learning hybrid automata which use discrete variables to identify system modes and, in the second step, they learn continuous models for each mode (Niggemann et al., 2012, Maier, 2014). This approach neglects the dependencies between discrete and continuous signals within each of the modes. Other approaches treat both types of signals in one learning step, for example by using suitable neural networks (Goh et al., 2017, Eiteneuer and Niggemann, 2018). However, they never discretize the result and, therefore, fail to identify the causalities and timings.
Discretizing the resulting model, for instance by turning it into a timed automaton in this work, leads to other advantages as well. First of all, the result can be understood by a human expert more easily. Furthermore, it helps to solve the classical stability–flexibility problem of machine learning: the automata are kept stable until the states are explicitly modified, for example when error thresholds are reached or by means of user interaction.
The following research questions are addressed in this work:
(RQ i) What is a reasonable definition of a discretized system state? Here, a latent representation of a probabilistic model and the capability of predicting signals is used to derive the states.
(RQ ii) How can the information from both the discrete and continuous part be handled equally and within one algorithmic framework? The proposed model is based on a deep network of stacked restricted Boltzmann machines. The input network layer represents raw continuous observations (time series), while the top layer is a distributed binary vector representation of a high level of abstraction which is then treated jointly and equally as originally discrete observations.
(RQ iii) How can we turn the result into a timed hybrid automaton which explicitly represents states, timing and signal dependencies? The most abstract network layer defines states and produces events, and is, therefore, interpreted as an automaton. It also handles the events from discrete data, which enables what we call the holistic timing and state analysis.
(RQ iv) The resulting automaton must be verified. This work introduces an anomaly-detection-based verification approach: any rejected automaton path implies an anomaly. There is no ultimate verification procedure of automaton correctness, but anomaly detection (AD) allows the analysis of a link between model complexity and AD performance. Here, two cases are studied: a simulation and a real-world conveyor system.
The structure of the paper is the following. In Section 2, a discussion of state-of-the-art approaches is given. The DENTA is defined and theoretically analyzed in Section 3. Section 4 provides an experimental evaluation of the approach, while the conclusion and future work are given in Section 5.
Section snippets
State-of-the-art approaches
In general, there are various methods for the discretization of continuous observations (Dougherty et al., 1995, Dimitrova et al., 2010). The simplest are the equal width interval and equal frequency interval described in Appendix C. For complex multidimensional problems, such approaches might not be useful as they do not consider the dependencies between the variables and thus cannot represent the structure of the data well.
However, machine learning approaches often yield better results.
Demonstration example
Consider a vertical conveyor transporting items from position A to B and vice versa. A central controller communicates with other components in a broader system and controls the movement between these two positions. The conveyor can be immobile at the top or bottom position, or it can move upwards or downwards. Furthermore, it can carry an item or not. Let us assume that the behavior of the conveyor at this abstract level can be described using a timed automaton with eight states (Fig. 1). The
Evaluation
To verify the learned automaton, we apply it to anomaly detection, i.e. we check whether the learned automaton can be used to detect non-normal timings, non-normal signal sequences and non-normal signal values. The anomaly detection procedure is described in Section 3. To evaluate the DENTA we compare it to the several other methods in terms of anomaly detection performance. These methods are shortly described in Appendix C. Two systems are used in the analysis: a real-world conveyor system is
Conclusion
The paper introduced a novel method called DENTA for the discretization of multivariate mixed-type time series into a timed automaton representation. It enables a holistic timing analysis of the discrete and continuous part, which can be essential for anomaly detection, diagnosis or the optimization of complex hybrid systems such as Cyber–Physical Production Systems (CPPSs). The construction of the automaton is based on a deep neural network comprised of stacked restricted Boltzmann machines
CRediT authorship contribution statement
Nemanja Hranisavljevic: Conceptualization, Methodology, Software, Data curation, Validation, Investigation, Resources, Writing - original draft. Alexander Maier: Methodology, Supervision, Writing - original draft, Writing - review & editing, Project administration, Funding acquisition. Oliver Niggemann: Conceptualization, Methodology, Investigation, Supervision, Validation, Writing - original draft, Writing - review & editing, Project administration, Funding acquisition.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was developed within the Fraunhofer Cluster of Excellence “Cognitive Internet Technologies”.
References (41)
- et al.
A theory of timed automata
Theoret. Comput. Sci.
(1994) - et al.
Fault detection, diagnosis and recovery using artificial immune systems: A review
Eng. Appl. Artif. Intell.
(2015) - et al.
Supervised and unsupervised discretization of continuous features
Cyber-physical production systems: Roots, expectations and R&D challenges
Procedia CIRP
(2014)- et al.
Industry 4.0: A bibliometric analysis and detailed overview
Eng. Appl. Artif. Intell.
(2019) - et al.
On-line adaptive clustering for process monitoring and fault detection
Expert Syst. Appl.
(2012) - et al.
State space neural networks and model-decomposition methods for fault diagnosis of complex industrial systems
Eng. Appl. Artif. Intell.
(2019) - et al.
Alarm management via temporal pattern learning
Eng. Appl. Artif. Intell.
(2017) - et al.
Evolving model identification for process monitoring and prediction of non-linear systems
Eng. Appl. Artif. Intell.
(2018) - et al.
Cyber-physical systems architecture for self-aware machines in industry 4.0 environment
Learning deep architectures for AI
Found. Trends Mach. Learn.
Automaton based on fuzzy clustering methods for monitoring industrial processes
Eng. Appl. Artif. Intell.
Model-based diagnosis of hybrid systems using satisfiability modulo theory
Discretization of time series data
J. Comput. Biol.
Industrial Internet: Pushing the Boundaries of Minds and MachinesTech. rep.
Unsupervised learning of distributions on binary vectors using two layer networks
A survey of fault diagnosis and fault tolerant techniques - part 1: Fault diagnosis with model-based approaches
IEEE Trans. Ind. Electron.
Cited by (12)
Deep learning-powered vessel traffic flow prediction with spatial-temporal attributes and similarity grouping
2023, Engineering Applications of Artificial IntelligenceLearning Physical Concepts in CPS: A Case Study with a Three-Tank System
2022, IFAC-PapersOnLineDevelopment of Fully Convolutional Neural Networks Based on Discretization in Time Series Classification
2023, IEEE Transactions on Knowledge and Data Engineering