Hammerstein Box-Jenkins System Identification of the Cascaded Tanks Benchmark System

Aljamaan, Ibrahim A.; Al-Dhaifallah, Mujahed M.; Westwick, David T.

doi:https://doi.org/10.1155/2021/6613425

Mathematical Problems in Engineering

On this page

Abstract Experimental Results Conclusions Data Availability Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2021 | Article ID 6613425 | https://doi.org/10.1155/2021/6613425

Hammerstein Box-Jenkins System Identification of the Cascaded Tanks Benchmark System

Ibrahim A. Aljamaan,¹Mujahed M. Al-Dhaifallah,²and David T. Westwick³

Academic Editor: Tirivanhu Chinyoka

Received07 Dec 2020

Accepted18 Feb 2021

Published28 Feb 2021

Abstract

A common process control application is the cascaded two-tank system, where the level is controlled in the second tank. A nonlinear system identification approach is presented in this work to predict the model structure parameters that minimize the difference between the estimated and measured data, using benchmark datasets. The general suggested structure consists of a static nonlinearity in cascade with a linear dynamic filter in addition to colored noise element. A one-step ahead prediction error-based technique is proposed to estimate the model. The model is identified using a separable least squares optimization, where only the parameters that appear nonlinearly in the output of the predictor are solved using a modified Levenberg–Marquardt iterative optimization approach, while the rest are fitted using simple least squares after each iteration. Finally, MATLAB simulation examples using benchmark data are included.

1. Introductions

Recently, system identification attracted the attention of many researchers and practitioners because of the difficulty in modeling many systems using physical modeling approaches. Good empirical mathematical models that represent a variety of practical applications in different engineering fields are required to address various needs [1]. These needs might be to understand and analyze the limitations of current systems, predict and simulate new experiments, or design new or modified controllers. The process of creating a mathematical model from measured data is called system identification [2, 3]. Unlike most machine learning methods, system identification gives more insight about the structure and dynamics of the system. A very common approach in system identifications is by using model structure methods, which is applied in this work. Unlike linear system identification methods that are well established and extensively applied in the literature, nonlinear system identification needs more attention because our real-life applications are nonlinear. The uniqueness of each nonlinearity increases the complexity of nonlinear systems and their identification [4–11].

There are a few attempts to apply nonlinear system identification techniques to common applications in different engineering fields reported in the literature. In the area of chemical process control, Aljamaan et al. [12] identified the dynamics of the continuously stirred tank reactor (CSTR) in the presence of nonstationary disturbances, Zhu [13] identified a distillation column, and Birpoutsoukis et al. [14] estimated a Volterra series model of the cascaded two-tank system. Similarly, in biomedical engineering applications, Vlaar et al. [15] identified brain signals by electroencephalography (EEG) while Cescon et al. [16] identified models of the blood sugar level inside the human body.

This work is limited to the cascaded two-tank system, where different identification approaches have been proposed in the literature. Unlike the nonlinear state-space method proposed by Relan et al. [1], Birpoutsoukis et al. [14] developed a nonparametric Volterra series-based approach. In this latter method, cancelling the undesired transient part played an important role in minimizing the error. Svensson and Schön [17] proposed a black-box nonlinear system identification state-space technique and applied it to cascaded tanks datasets. Different system identification approaches were proposed in the literature such as [18–32].

In this paper, a prediction error minimization (PEM) nonlinear system identification parametric approach, similar to the one proposed in [10], is applied to approximate the two-tank cascaded model, where the benchmark data provided by Schoukens et al. [26] were used. The main contribution is to use such general nonlinear model structure, Hammerstein Box-Jenkins model, and separable least squares (SLS) identification approach to predict the parameters of cascaded tanks system. The suggested model structure to fit this application is Hammerstein Box-Jenkins model, where a static nonlinearity is in series with a linear dynamic filter whose output is summed with an autoregressive moving average (ARMA) noise model. Separable least squares (SLS) techniques are suggested, where second-order gradient descent-based optimization approach is performed on the nonlinear parameters while ordinary least squares regression is used, after each iteration, to estimate the coefficients that appear linearly in the predictor.

The arrangement of the paper is as follows. After a physical description of the two-tank system and approximated model structure in Section 2, Section 3 includes the identification suggested approach. Then, simulation examples and results’ discussion are illustrated. Finally, concluding remarks are provided in the conclusion.

2. Problem Definition

2.1. The Cascaded Water Tanks

The cascaded water tanks system is a benchmark system proposed in [26]. It comprises two water tanks, with free outlets as shown in Figure 1. The first tank is fed by a pump and drains into the second tank, which itself drains into the reservoir (see Figure 2). The input to the system is the voltage applied to the pump, while the output is the reading on a level sensor in the second tank. Note that if the upper tank overflows, some of the overflow will land in the second tank, while the rest will go directly into the reservoir. The source of nonlinearity is mainly at the input signal and combined with a small nonlinearity during the measurement [26].

2.2. The Proposed Model Structure

The Box-Jenkins model is, in some sense, ideal for the modeling and identification of linear time invariant (LTI) systems because many models can be regarded as special cases such as the AR, MA, ARMA, ARX, ARMAX, and OE models [33]. It includes separate filters for the system and noise models, which gives it a great deal of flexibility. Replacing the linear process filter in the Box-Jenkins model with a nonlinear model produces a much more flexible model, as it can model nonlinear behaviour in the plant but can still provide accurate predictions, even when the stochastic disturbance is highly autocorrelated. In this section, we consider the Hammerstein Box-Jenkins model, shown in Figure 3. It combines the linear dynamics of a Box-Jenkins model with the nonlinear structure of a Hammerstein model. The main reason for selecting Hammerstein is the previous knowledge of the differential balance equations for the two-tank system that imply the main nonlinearity is at the input which is in accordance with the Hammerstein model. Different system identification approaches are presented in [34–40].

The main identification target is to estimate the process and noise parameters and the expansion coefficients of the nonlinearity. An important factor in achieving an acceptable identification result is a good initial guess. This is because when any iterative optimization method is used, it is uncertain that the assessment function will converge to the global minimum unless the initial guess of the parameter vector is close enough. The model is formulated aswhere is the output of the deterministic part of the system and the noise part is

The deterministic process output side is computed bywhere is the output of the static nonlinearity:

In this paper, the memoryless nonlinearity will be approximated by a polynomial of degree M because of the efficiency and ability to apply LS to find the coefficients of the polynomial nonlinearity.

3. Preliminary

3.1. Prediction Error Method (PEM)

The prediction error method (PEM) searches for the parameter vector, , that reduces the difference between the measured output and the output prediction generated by the model. Thus, ε is the prediction error.

Note that the parameter vector, , consists of the elements of the process and noise filters and coefficients of the polynomial nonlinearity.

3.1.1. One-Step Ahead Prediction

The most common option in PEM is the one-step ahead prediction, where the existing output, , at time is approximated given all I/O measurements up to including . The one-step ahead predictor equation for the Hammerstein Box-Jenkins model is as follows:where the noise model predictor, , is solved by

By substituting (3) and (8) in (7):

The main objective, (10), is to compute the parameter vector that minimizes the mean squared error (MSE) of the cost function, , in (11).

3.2. Separable Least Squares

In the SLS algorithm, the parameter vector, , is divided into two sets: a group of parameters that appears linearly in the prediction error, , and the remaining parameters, [41]. The prediction error is generally a nonlinear function of ; although in some cases, one or more elements of may appear bilinearly with elements of . In these cases, the decision as to which parameter is “linear” or “nonlinear” is arbitrary. Thus, the parameter vector is subdivided as

For the Hammerstein model, the prediction error can be treated as linear in either the polynomial coefficients or the plant numerator. The output of process model, , and polynomial is formulated as follows:

The parameter vector of the linear part, , is estimated by doing simple LS.where is the vector that contains all experimental results of . The nonlinear parameter vector, , is estimated using an iterative approach.

It is noticed from (16) that relied on in finding the direction of the search. Similar to the parameter vector , the Jacobian is divided into two parts too: and that consist of the linear parameters and nonlinear coefficients, respectively:and formulated in (18) and (19):

The Jacobian of the SLS is computed by

Because the predictor output relies on the nonlinear parameters that might result variations in the linear elements the projection of the linear part, , defined in (21), is required to calculate this variation.

The complementary projection, , is defined in the following equation:

Now can be calculated by

Once has been computed, can be updated using any quasi-Newton optimization, and Levenberg–Marquardt is proposed in this work. The separable least squares approach is described in more detail by Aljamaan et al. [12].

The nonlinear vector parameters can be updated using any quasi-Newton optimization.

3.3. L-M Optimization

The Levenberg–Marquardt algorithm is a second-order iterative optimization-based technique, where the update applied to the parameter depends on the local gradient and the approximation of the error surface's local curvature [42, 43]. It is applied to find the nonlinear LS problems, where minimum is explored, but it is not guaranteed to be the global minimum. Also, it is used in software applications to fit best curve. The phase direction and length were measured together in the L-M iteration. The update is defined in (14).

Note that the distance and the direction of the step are chosen so that the cost decreases.

The L-M method formulas are as follows:where ε is the error and J is the Jacobian. d_(k) is the updated step to control the step size and also responsible for speeding up of the convergence and ensures the stability, and is the approximate Hessian; the step size for the Hessian approximation is included. IM is the M × M identity Matrix.

3.4. Instrumental Variable (IV)

The global minimum can be always achieved by using LS if the parameters of the model are linear. On the other hand, models that are nonlinear in their parameters require iterative optimization techniques to converge to a minimum. This minimum may not be the global minimum and hence the importance of a good initial guess. Moreover, predicting an excellent initial guess plays a significant factor to have a successful identification. The IV approach, used in this work, is a familiar option that yields unbiased approximated coefficients (this is explained in [2, 3]). An extended IV approach is provided in [44].

3.5. Hammerstein Box-Jenkins Model Identification Using SLS

In the following, a summary of the proposed SLS identification algorithm (Algorithm 1) is presented.

(1)	Find initial guess using IV method.
(2)	Compute using (5), and then fit using LS.
(3)	Find and as in (18) and (19).
(4)	Calculate the updated step , (26), and then update, , (16).
(5)	Compute , (5), using updated , and fit .
(6)	Using LS.
(7)	Compute the error and cost function, and , respectively.
(8)	If the cost decreases.
(9)	If is significant Then
(10)	Exit
(11)	else increase speed of convergence by setting the update step , then go back to step 3
(12)	End
(13)	else the cost, , increases Then
(14)	Decrease the speed of convergence by setting: and go back to step 5
(15)	end

4. Experimental Results

MATLAB simulation examples using two datasets of 1024 points each, identification and validation data, were provided by Vlaar et al. [15]. These datasets were collected from the cascade tanks system shown in Figure 2. The input u (t) is a multisine signal, which is the voltage applied to pump the water from the reservoir to the upper tank with frequency range between 0 and 0.0144 Hz and sampling time Ts equal to 4 seconds. The water level of the lower tank is the output of the system, y (t), with signal to noise ratio, SNR, approximately equal to 40 dB. Input and output are shown in Figure 4. More information about the data is included in [15].

A Hammerstein Box-Jenkins model structure similar to the illustrated one in Figure 3 is applied to identify the approximated mathematical model. A third-order polynomial nonlinearity in cascade with second-order filter that represents the plant model is selected. Unlike the process model, the noise subsystem was of first-order ARMA filter.

The order was selected manually using the training data. On the other hand, Table 1 illustrates the MSE results using the validated dataset. MSE function was chosen as the assessment tool. The first experiment was applied by selecting a first-order process and noise model with zero-order nonlinearity, and it produces high MSE results. Then, the order of the nonlinearity increases and it is clear that the cost decreases in the 2^nd and 3^rd nonlinearity cases. Unlike in the 2^nd and 3^rd cases, there is very small difference in the cost between 3^rd and 4^th order nonlinearities. After that, the process linear dynamic order was increased, and it was noticed that using 2^nd order plant model and 3^rd order nonlinearity produces the minimum cost. Notice that a 1^st order noise model was fixed for all the experiments.

The proposed parametric identification method based on Section 3 was applied. Figure 5 shows the difference between the simulated outputs, dotted black lines, that tracks the measured output represented in red solid line. Moreover, the cost function using iterative second order gradient descent method based is plotted in Figure 6, where the final convergence was at iteration 25. The nonlinearity is illustrated in Figure 7. The mean square error, MSE, of the 2^nd order linear dynamic subsystem, the 1^st order noise filter, and 3^rd order nonlinearity was the minimum with 0.0073.

A comparison between the results obtained by applying this proposed approach and two previous studies using the same benchmark data by Relan et al. and Birpoutsoukis et al. [1, 14] is given in Table 2 and Figure 8. It is clear that the proposed Hammerstein Box-Jenkins approach outperformed the other algorithms and has the least MSE value.

(a)

(b)

(c)

5. Conclusions

In this research, a nonlinear identification approach is applied to estimate a cascade tank system from data provided on the nonlinear system identification benchmark website as one of the challenging datasets, provided by Aljamaan[11]. The suggested model structure that fits this application is a Hammerstein Box-Jenkins model. This parametric method is prediction error based, where the linear dynamic process and noise subsystem elements are approximated using iterative Hessian method optimization technique; on the other hand, the polynomial nonlinearity coefficients are fitted using the simple LS method after each iteration because they appear linearly in the predictor. In order to prevent the ill-condition issue and increase the chance of convergence, an IV method is used for the initial guess. Different orders were tested to select the suitable one, and simulated data were applied to validate the results. The final results are promising for the successful identification of real two-tank systems.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

All authors collaborated and contributed equally to this study. All authors have read and agreed to the published version of the manuscript.

Acknowledgments

This research was funded by the Deanship of Scientific Research (DSR), Imam Abdulrahman Bin Faisal University, under grant no. 2020-110-Eng.

References

R. Relan, K. Tiels, A. Marconato, and J. Schoukens, “An unstructured flexible nonlinear model for the cascaded water-tanks benchmark,” in Proceeding of the 20th World Congress IFAC (The International Federation of Automatic Control) Conference, pp. 454–459, Toulouse, France, July 2017.
View at: Google Scholar
L. Ljung, System Identification Theory for the User, Prentice-Hall, Upper Saddle River, NJ, USA, 1999.
T. Söderstrom and P. Stoica, System Identification, Prentice-Hall, Upper Saddle River, NJ, USA, 1989.
R. Pintelon and J. Schoukens, System Identification A Frequency Domain Approach, Wiley-IEEE Press, Piscataway, NJ, USA, 2012.
F. Giri and E. Bai, Block-oriented Nonlinear System Identification, Springer, Berlin, Germany, 2010.
D. Westwick and R. Kearney, Identification of Nonlinear Physiological System, Wiley-IEEE Press, Piscataway, NJ, USA, 2003.
J. Schoukens and L. Ljung, “Nonlinear system identification: a user-oriented road map,” IEEE Control Systems Magazine, vol. 39, no. 6, pp. 28–99, 2019.
View at: Google Scholar
O. Nelles, Nonlinear System Identification, Springer-Verlag, New York, NY, USA, 2001.
K. J. Åström and P. Eykhoff, “System identification-a survey,” Automatica, vol. 7, no. 2, pp. 123–162, 1971.
View at: Publisher Site | Google Scholar
A. K. Tangirala, Principles of System Identification: Theory and Practice, CRC Press, Boca Raton, Fl, USA, 2014.
I. Aljamaan, “Nonlinear identification of two different closed-loop models using separable least squares methods,” Master’s Dissertation, University of Calgary, Calgary, Canada, 2011, http://library.ucalgary.ca/.
View at: Google Scholar
I. Aljamaan, D. Westwick, and M. Foley, “Non-linear closed-loop identification of CSTR in the presence of a non-stationary disturbances,” IFAC-PapersOnLine, vol. 48, no. 28, pp. 1029–1034, 2015.
View at: Publisher Site | Google Scholar
Y. Zhu, “Distillation column identification for control using wiener model,” in Proceedings of the American Control Conference, vol. 5, pp. 3462–3466, San Diego, CA, USA, June 1999.
View at: Google Scholar
G. Birpoutsoukis, P. Z. Csurcsia, and J. Schoukens, “Nonparametric Volterra series estimate of the cascaded water tanks using multidimensional regularization,” in Proceedings of the 20th World Congress IFAC (The International Federation of Automatic Control) Conference, pp. 478–483, Toulouse, France, July 2017.
View at: Google Scholar
M. P. Vlaar, G. Birpoutsoukis, J. Lataire et al., “Modeling the nonlinear cortical response in EEG evoked by wrist joint manipulation,” in Proceedings of the, vol. 26, no. 1, pp. 205–215.
View at: Publisher Site | Google Scholar
M. Cescon, F. Ståhl, M. Olsson, and R. Johansson, “Subspace-based model identification of diabetic blood glucose dynamics,” in Proceedings of the 15th IFAC Symposium on System Identification, pp. 233–238, Saint-Malo, France, March 2009.
View at: Google Scholar
A. Svensson and T. B. Schön, “A flexible state-space model for learning nonlinear dynamical systems,” Automatica, vol. 80, pp. 189–199, 2017.
View at: Publisher Site | Google Scholar
J. Belz, T. Münker, T. Heinz, G. Kampmann, and O. Nellez, “Automatic modeling with local model networks for benchmark processes,” in Proceedings of the 20th World Congress IFAC (The International Federation of Automatic Control) Conference, pp. 472–477, Toulouse, France, July 2017.
View at: Google Scholar
M. Brunot, A. Janot, and F. Carrillo, “Continuous-time nonlinear systems identification with output error method based on derivative-free optimisation,” in Proceedings of the The 20th World Congress IFAC (The International Federation of Automatic Control) Conference, pp. 466–471, Toulouse, France, July 2017.
View at: Google Scholar
M. Schoukens and J. Noël, “Three benchmarks addressing open challenges in nonlinear system identification,” in Proceedings of the the 20th World Congress IFAC (The International Federation of Automatic Control) Conference, pp. 448–453, Toulouse, France, July 2017.
View at: Google Scholar
H. Zhou, C. Ibrahim, and W. Pan, “A sparse bayesian deep learning approach for identification of cascaded tanks benchmark,” 2019, https://arxiv.org/abs/1911.06847.
View at: Google Scholar
M. H. D. M. Ribeiro, V. C. Mariani, L. Manke, and L. S. Coelho, “Nonlinear system identification based on ensemble learning: the cascaded tanks system case,” in Proceedings of the 14th Brazilian Symposium on Intelligent Automation (SBAI), pp. 415–420, Ouro Preto, Brazil, October 2019.
View at: Google Scholar
G. Giordano and J. Sjöberg, “Black-and white-box approaches for cascaded tanks benchmark system identification,” Mechanical Systems and Signal Processing, vol. 108, pp. 387–397, 2018.
View at: Publisher Site | Google Scholar
P. Mattsson, D. Zachariah, and P. Stoica, “Identification of cascade water tanks using a PWARX model,” Mechanical Systems and Signal Processing, vol. 106, pp. 40–48, 2018.
View at: Publisher Site | Google Scholar
J. G. Stoddard and J. S. Welsh, “Regularized basis function estimation of Volterra kernels for the cascaded tanks benchmark,” IFAC-PapersOnLine, vol. 51, no. 15, pp. 413–418, 2018.
View at: Publisher Site | Google Scholar
M. Schoukens, P. Mattson, T. Wigren, and J.-P. Noël, “Cascaded tanks benchmark combining soft and hard nonlinearities,” in Proceedings of the Conference: ERNSI 2015, Varberg, Sweden, September 2015, http://homepages.vub.ac.be/∼7Emschouke/benchmark2016.html.
View at: Google Scholar
T. Wigren and J. Schoukens, “Data for benchmarking in nonlinear system identification,” Tech. Rep., Technical Reports, Department of Information Technology 2013-006, Uppsala, Sweden, 2013, http://www.it.uu.se/research/publications/reports/2013-006/SNLA80mVZipped.zip.
View at: Google Scholar
B. Stenlund and A. Medvedev, “Level control of cascade coupled flotation tanks,” IFAC Proceedings Volumes, vol. 33, no. 22, pp. 185–190.
View at: Publisher Site | Google Scholar
T. Wigren, “Recursive prediction error identification and scaling of non-linear state space models using a restricted black box parameterization,” Automatica, vol. 42, no. 1, pp. 159–168, 2006.
View at: Publisher Site | Google Scholar
T. Wigren, “MATLAB software for recursive identification and scaling using a structured nonlinear black-box model—revision 2,” Technical reports 2005-022, Department of Information Technology, Uppsala University, Uppsala, Sweden, 2005, http://www.it.uu.se/research/reports/.
View at: Google Scholar
T. Wigren, Input-output Data Sets for Development and Benchmarking in Nonlinear Identification, Department of Information Technology, Uppsala University, Uppsala, Sweden, 2010, Technical Reports.
K. Hamid and M. Fadali, “Nonlinear system identification using neural networks and trajectory-based optimization,” 2018, https://arxiv.org/abs/1804.10346.
View at: Google Scholar
G. Box and G. Jenkins, Time Series Analysis Forecasting and Control, Holden Day, San Francisco, CA, USA, 1970.
I. A. D. Westwick and M. Foley, “Prediction error identification of Hammerstein models in the presence of ARIMA disturbances,” in Proceedings of the IEEE Conference on Control Applications (CCA), Juan Les Antibes, France, October 2014.
View at: Google Scholar
J. Sjöberg and M. Viberg, “Separable nonlinear least squares minimization-possible improvements for neural net fitting,” IEEE Workshop on Neural Networks for Signal Processing, vol. 7, pp. 345–354, 1997.
View at: Google Scholar
E.-W. Bai and D. Li, “Convergence of the iterative Hammerstein system identification algorithm,” IEEE Transactions on Automatic Control, vol. 49, no. 11, pp. 1929–1940, 2004.
View at: Publisher Site | Google Scholar
D. T. Westwick and R. E. Kearney, “Separable least squares identification of nonlinear Hammerstein models: application to stretch reflex dynamics,” Annals of Biomedical Engineering Society, vol. 29, no. 8, pp. 707–718, 2001.
View at: Publisher Site | Google Scholar
E. J. Dempsey and D. T. Westwick, “Identification of Hammerstein models with cubic spline nonlinearities,” IEEE Transactions on Biomedical Engineering, vol. 51, no. 2, pp. 237–245, 2004.
View at: Publisher Site | Google Scholar
K. Narendra and P. Gallman, “An iterative method for the identification of nonlinear systems using a Hammerstein model,” IEEE Transactions on Automatic Control, vol. 11, no. 3, pp. 546–550, 1966.
View at: Publisher Site | Google Scholar
S. A. Billings and S. Y. Fakhouri, “Identification of systems containing linear dynamic and static nonlinear elements,” Automatica, vol. 18, no. 1, pp. 15–26, 1982.
View at: Publisher Site | Google Scholar
A. Ruhe and P. Å. Wedin, “Algorithms for separable nonlinear least squares problems,” SIAM Review, vol. 22, no. 3, pp. 318–337, 1980.
View at: Publisher Site | Google Scholar
A. Ranganathan, “The levenberg-marquardt algorithm,” Tutoral on LM Algorithm, vol. 11, no. 1, pp. 101–110, 2004.
View at: Google Scholar
J. Moré, “The Levenberg-Marquardt algorithm: implementation and theory,” in Numerical Analysis, pp. 105–116, Springer, Berlin, Heidelberg, 1978.
View at: Google Scholar
V. Laurain, M. Gilson, H. Garnier, and P. Young, “Re-fined instrumental variable method for identification of Hammerstein continuous-time box-jenkins models,” in Proceedings of the 47th IEEE Conference on Decision and Control, pp. 1386–1391, Cancun, Mexico, December 2008.
View at: Google Scholar

Copyright

Copyright © 2021 Ibrahim A. Aljamaan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

539

Downloads

866

Citations