Machine-learning-based state estimation and predictive control of nonlinear processes
Introduction
Closed-loop performance of chemical processes under model-based controllers (e.g., model predictive control (MPC)) depends on the model representation of the process, and the availability of real-time state measurements. In general, MPC uses a first-principles model or a data-driven process model to predict state evolution in the optimization problem, and adjusts its control actions with state feedback from the sensor measurements. However, measurements of key process states such as species concentration in a chemical reactor could be time-consuming and sometimes involves manual manipulation of samples during offline protocols (McKenna et al., 2000, Zambare et al., 2002). Additionally, the cost of equipment for getting the targeted measurement in real time also hinders its real-time application in chemical plants (Patwardhan et al., 2012). One way to address this issue is to combine measurable process state variables (e.g., pressure, level, and temperature measurements) with state estimation techniques to predict unmeasured states in real-time operation.
State estimation has been extensively studied in the literature, and includes methods for both deterministic and stochastic cases (Radke and Gao, 2006, Dochain, 2003, Patwardhan et al., 2012, Alexander et al., 2020). In stochastic state estimation, many methodologies have been proposed including recursive and optimization-based approaches, which can also address the constrained and unconstrained estimation problems. Extended Kalman filter (EKF) is one of the most popular recursive methods for unconstrained nonlinear systems. Moving horizon estimator is an optimization-based methodology that can account for constraints in its formulation. Additionally, other methodologies such as unscented Kalman filter, particle filter, constrained version of the EKF, and combination of the above methods, have been proposed to improve the performance of EKF (e.g., Lima and Rawlings, 2011, Patwardhan et al., 2012, Alexander et al., 2020). In deterministic state estimation, Luenberger-based observers are common estimation methods for the practitioners (Dochain, 2003, Ali et al., 2015). Additionally, extended Luenberger observer, sliding mode observer, adaptive state observer, high-gain observer, geometric observer, backstepping observer have found diverse applications in many fields (e.g., Ali et al., 2015). Similarities and differences among the above methods and their advantages and disadvantages are further discussed in Radke and Gao (2006) and Ali et al. (2015). In order to achieve a desired performance using these methodologies, a mathematical model for the targeted system is generally needed to describe process dynamics in a certain operating region. However, the development of such a process model for some complex reacting systems using first-principles knowledge could be challenging. For example for a catalytic carbon monoxide oxidation over Pt-alumina, a common Langmuir-Hinshelwood rate law is only valid in a small region of operation (Porru et al., 2000).
Machine learning has recently attracted an increasing level of attention in process modeling. Among many different machine learning methods, recurrent neural networks (RNN) and long-short-term-memory (LSTM) networks have been utilized to model dynamic systems due to their temporal dynamic behavior. Additionally, hybrid modeling that relies on both first-principles knowledge and process operational data can also be used to model nonlinear chemical processes and is one of the most interesting and challenging problems in the data science era (Venkatasubramanian, 2019). The idea of hybrid modeling is to use the best features of first-principles model (i.e., parametric models) and of data-driven models (i.e., nonparametric models) to better capture the process dynamics. There are many examples of hybrid modeling and their applications to chemical engineering problems in the literature, e.g., (Porru et al., 2000, Oliveira, 2004, Von Stosch et al., 2014, Zendehboudi et al., 2018, Bangi and Kwon, 2020, Lee et al., 2020). For example, in Bangi and Kwon (2020), a hybrid model was developed for a hydraulic fracturing process, where the process first-principles model was integrated with a deep neural network. In Porru et al. (2000), a neural network model was developed to represent the reaction kinetics and was coupled with the first-principles model to obtain a hybrid model that was successfully applied within the EKF. It is reported that hybrid models can not only augment the region of operation, but also provide a more general modeling framework that can build models faster and need no process insights (Wilson and Zorzetto, 1997).
Machine learning models can be utilized in model-based controllers to predict future states. Recently, in Wu et al., 2019a, Wu et al., 2020, machine-learning-based MPC schemes have been proposed to optimize process performance and ensure system stability with feedback measurements of process state variables assumed to be available. However, the assumption of availability of full state measurements for feedback control may not hold for the chemical processes with state variables difficult to measure in real time. In this work, we propose two machine learning approaches: (a) recurrent neural networks, and (b) hybrid models using feed-forward neural networks and first-principles models, to model nonlinear processes. Then, we integrate the RNN model and the hybrid model within the extended Luenberger observer framework and develop Lyapunov-based MPC using state estimates from machine-learning-based state estimators. Specifically, Section 2 introduces the preliminaries, including the class of systems, and the formulation of extended Luenberger observer. Section 3 presents the formulation of RNN models and of the RNN-based Luenberger observer. Section 4 presents the formulation of hybrid models and of the hybrid-model-based state estimator. Section 5 presents the formulation of output feedback model predictive controller that uses state estimates from the aforementioned machine-learning-based state estimators. Finally, in Section 6, a chemical reactor example is used to illustrate the effectiveness of the proposed estimation approaches.
Section snippets
Notations
The Euclidean norm of a vector is represented by . The standard Lie derivative is represented as . The notation stands for set subsection, i.e., . The function is said to be of class if it is continuously differentiable.
Class of systems
We consider the following class of continuous-time nonlinear systems in state-space form:where the state vector is , the output vector is , and the input vector is
Recurrent neural network (RNN)
As a process model is needed in the extended Luenberger observer of Eq. (2). The following RNN model is developed to approximate the nonlinear system of Eq. (1) using process operational data when a first-principles model is not available:where is the RNN state vector, and is the manipulated input vector. is a vector of both and , where is the nonlinear activation function. is a
Hybrid-model-based state estimator
In this section, we introduce a state estimator designed based on a hybrid model that integrates feed-forward neural network (FNN) with first-principles model. In this case, the FNN model is only used to approximate the nonlinear terms in Eq. (1), while the first-principles model of Eq. (1) can be derived from physical laws such as mass and energy balances.
Output feedback model predictive control
In this section, an output feedback model predictive control (MPC) is designed based on state estimates provided by the RNN-based estimator to stabilize the nonlinear system of Eq. (1) at the steady-state. Specifically, the Lyapunov-based MPC is used in this work and the formulation is presented as the following optimization problem:
Application to a chemical reactor example
In this section, a nonlinear chemical process is used to illustrate the application of the proposed RNN-based and hybrid-model-based estimators in the LMPC controller. A non-isothermal, a well mixed continuous stirred tank reactor (CSTR) is considered, with the following reversible first-order exothermic reaction (Zhang et al., 2019):The nonlinear dynamical model that describes the process dynamics is given by the following mass and energy balance equations:
Conclusion
In this work, we proposed machine-learning-based state estimation approaches for nonlinear processes. The RNN model was first developed to represent process dynamics in the operating region, and incorporated in extended Luenberger observer. Then, the RNN-based estimator was used to provide state estimates for the optimization problem of LMPC. Subsequently, a hybrid model was develop to represent process dynamics and used in the state estimator. From closed-loop simulations, it was demonstrated
Declaration of Competing Interest
The authors report no declarations of interest.
Acknowledgments
Financial support from the National Science Foundation and the Department of Energy is gratefully acknowledged. Mohammed Alhajeri would like to express his sincere appreciation to Kuwait University for its support through the KU-scholarship program. Fahad Albalawi acknowledges Taif University for their support via Taif University Researchers Supporting Project (TURSP-2020/97).
References (36)
- et al.
Deep hybrid modeling of chemical process: application to hydraulic fracturing
Comput. Chem. Eng.
(2020) State and parameter estimation in chemical and biochemical processes: a tutorial
J. Process Control
(2003)- et al.
An encoder-decoder LSTM-based EMPC framework applied to a building HVAC system
Chem. Eng. Res. Des.
(2020) - et al.
Local and global sensitivity analysis for a reactor design with parameter uncertainty
Chem. Eng. Res. Des.
(2004) - et al.
Integrating dynamic neural network models with principal component analysis for adaptive model predictive control
Chem. Eng. Res. Des.
(2020) - et al.
State and disturbance estimation for nonlinear systems affine in the unmeasured variables
Comput. Chem. Eng.
(1998) - et al.
A universal formula for stabilization with bounded controls
Syst. Control Lett.
(1991) - et al.
Feedback linearizing control using hybrid neural networks identified by sensitivity approach
Eng. Appl. Artif. Intell.
(2005) - et al.
A comparison of nonlinear observers for output feedback model-based control of seeded batch crystallization processes
J. Process Control
(2011) Combining first principles modelling and artificial neural networks: a general framework
Comput. Chem. Eng.
(2004)