Elsevier

Automatica

Volume 129, July 2021, 109663
Automatica

Brief paper
A novel reduced-order algorithm for rational models based on Arnoldi process and Krylov subspace

https://doi.org/10.1016/j.automatica.2021.109663Get rights and content

Abstract

This paper presents a novel reduced-order algorithm for identifying rational models. From the Arnoldi process, an orthonormal basis of the Krylov subspace is constructed. Based on the Krylov subspace, a high-order cost function is transformed into a low-order one, thereby significantly reducing the computational efforts. This algorithm can be considered as a reduced-order least squares (LS) algorithm or an extension of the traditional gradient iterative (GI) algorithm for different Krylov subspaces, and it presents several advantages over the traditional LS and GI algorithms. The simulated numerical results/figures are consistent with the analytically derived results in terms of the feasibility and effectiveness of the proposed algorithm.

Introduction

Many dynamic systems can be modeled using various nonlinear models, e.g., bilinear models (Andrianasolo et al., 2017, Zhang and Yang, 2019), generalized exponential autoregressive models, and rational models (Chen and Gan, 2018, Chen et al., 2020, Geng et al., 2019). The rational model is mathematically expressed by a ratio of two polynomials, and has several characteristics that the other linear and nonlinear models do not have (Billings & Zhu, 1991). For example, (1) almost all the other smooth linear and nonlinear models can be regarded as its subsets; (2) its structure can be more concise than a polynomial expansion; (3) it can be frequently used to represent complicated system structures with a fairly low degree (Zhu et al., 2015, Zhu et al., 2017). Owing to these advantages, rational models have been extensively studied in the last few decades.

This study focuses on the identification of rational models with the aim of reducing the computational efforts and avoiding the matrix eigenvalue calculation in the parameter estimation procedure when the order of the rational model is high. Rational models are widely used in engineering practices, such as chemical processes and biological reactions (Chen et al., 2018c, Kamenski and Dimitrov, 1993, Mu et al., 2017). However, the research on the identification of rational models is still in its early stage. Although various algorithms have been developed and validated with simulations and applications, rigorous analytical descriptions and reports are still rare in the literature. Existing methods for rational model identification are roughly categorized into two types: the least squares (LS) and gradient iterative (GI) algorithms. These two types of algorithms can estimate the parameters of the rational models, but they also introduce some challenges, e.g., the heavy computational efforts and the matrix eigenvalue calculation for high-dimensional matrices.

The LS algorithm aims to find the optimal parameter estimates that minimize a cost function J(θ), with collected input–output data {u(1),,u(L),y(1),,y(L)} (Chen et al., 2019, Ruscio, 2000, Zhao et al., 2019). Its basic idea is to minimize the cost function in the sense of a 2-norm. Recently, many LS algorithms have been applied for the identification of rational models. In (Billings & Zhu, 1991), an extended linear LS algorithm was proposed for rational models, which is simpler than the nonlinear LS algorithm owing to its concise cost function. In (Zhu, 2005), Zhu proposed an implicit LS algorithm for a nonlinear rational model, where unbiased parameter estimates can be obtained using the implicit LS algorithm. Chen et al. developed a biased compensation least squares based threshold algorithm for time-delay rational models. The algorithm estimates the time delay and the parameters using the redundant rule (Chen, Zhu, & Li, 2018d). The inverse matrix calculation in the LS algorithm leads to heavy computational costs, especially for large-scale systems.

The GI algorithm is another identification method for rational models, which avoids inverse matrix calculation. Thus, it requires less computational efforts. In general, the search direction and step-length are two main challenges in GI algorithm development (Liu and Lu, 2010, Magnusson et al., 2018, Wang et al., 2017, Wang and Tang, 2014). A better direction can increase the convergence rates, for example, the conjugate gradient method has an improved search direction (Abbasbandy, Jafarian, & Ezzati, 2005). A suitable step-length is also important for the GI algorithm. For example, a small step-length would lead to slow convergence rates, whereas a large step-length would diverge the result. Recently, many methods have been developed for step-length selection, such as the forgetting factor based GI algorithm (Xu, Wan, Ding, Alsaedi, & Hayat, 2019), modified GI algorithm (Ding, Shi, Wang, & Ding, 2010), multi-step GI algorithm (Chen, Ding, Liu, & Zhu, 2018a), and Aitken GI algorithm (Wang & Li, 2019). Although the GI algorithm requires no inverse matrix calculation and increases the convergence rates through a better search direction and a suitable step-length, it also brings a challenging issue: the calculation of the matrix eigenvalue, especially for a high-dimensional matrix. It is difficult and costly to obtain the roots of a high order equation; thus, the GI algorithm is often used for low-order system identification.

To avoid the eigenvalue calculation and to reduce the computational efforts, this study proposes a reduced-order algorithm for rational models. This algorithm is based on the Arnoldi process and Krylov subspace, which are widely used for solving large sparse linear systems of equations (Saad & Schultz, 1986). The Arnoldi’s method is used to generate an orthonormal matrix on the Krylov subspace Gk=span{v(1),Nv(1),,Nk1v(1)}. Then, an M×M inverse matrix calculation is reduced to a k×k (k<M) inverse matrix calculation. Because a small k may lead to poor estimation accuracy, an iterative based reduced-order algorithm that utilizes more iterations to improve the estimation accuracy is developed. In summary, the contributions of this study are as follows:

(1) Compared with the GI algorithm, there is no eigenvalue calculation in the reduced-order algorithm. Thus, this algorithm can be widely used for large-scale system identification. Furthermore, it has a faster convergence rate than that of the GI algorithm.

(2) Compared with the LS algorithm, the reduced-order algorithm performs a low-dimensional matrix inverse calculation rather than a high-dimensional one. Thus, it requires less computational efforts.

(3) Integration of the iterative algorithm and reduced-order method into a comprehensive framework, which can improve the estimation accuracy and establish a link between the GI and LS algorithms.

The remainder of this paper is organized into five sections. Section 2 describes the rational model and introduces the traditional LS and GI algorithms. Section 3 proposes a reduced-order algorithm for rational models. An iterative based reduced-order algorithm that improves the parameter estimation accuracy is proposed in Section 4. Section 5 tests the developed procedures using a simulation example and shows the exemplary procedures for potential users. Finally, the conclusion and future directions are presented in Section 6.

Section snippets

Traditional GI algorithm

Consider the following rational model, y(t)=ρ(t)+e(t)μ(t),where y(t) is the output, e(t) is a Gaussian white noise that satisfies e(t)N(0,σ2), ρ(t) and μ(t) are expressed as ρ(t)=φT(t)θa,μ(t)=ψT(t)θb, where the vectors φ(t) and ψ(t) are the products of past inputs {u(t1),u(t2),} and past outputs {y(t1),y(t2),}, such as y(t1)u(t2), y2(t2), and y2(t1)u(t1), and their structures are usually assumed to be known in prior (Billings and Zhu, 1991, Chen et al., 2018d, Mu et al., 2017, Zhu,

Reduced-order algorithm and its convergence analysis

To avoid the eigenvalue calculation and reduce the computational efforts, a reduced-order (RO) algorithm is developed.

Iterative based reduced-order algorithm

To reduce the computational costs, we should choose a small k; however, reducing k leads to less accurate parameter estimates. In this section, an iterative based RO (I-RO) algorithm is developed, which can successfully solve the problem.

Example

Consider the following rational model, y(t)=ρ(t)+e(t)μ(t),ρ(t)=0.2y(t1)+0.1y(t1)u(t1)+u(t1)+0.4y(t2)0.3y(t3)+0.7u(t2)+0.2y(t4)+0.6u(t3),μ(t)=1+y2(t1)+y2(t2). It can be simplified as, y(t)=0.2y(t1)+0.1y(t1)u(t1)+u(t1)+0.4y(t2)0.3y(t3)+0.7u(t2)+0.2y(t4)+0.6u(t3)y(t)y2(t1)y(t)y2(t2)+e(t). The input {u(t)} is taken as a persistent excitation signal sequence with zero mean and unit variance, {e(t)} is taken as a white noise sequence with zero mean and variance σ2=0.102, and

Conclusions

A reduced-order algorithm is proposed for estimating the parameters of rational models in this paper. The new algorithm is based on the Arnoldi process and Krylov subspace and establishes a link between the GI and LS algorithms. (1) When k=1, the I-RO algorithm can be regarded as a GI algorithm; (2) when k=M, the I-RO algorithm is equivalent to an LS algorithm; (3) when 1<k<M, the I-RO algorithm provides a middle ground between the LS and GI algorithms. Regarding the properties of the I-RO

Acknowledgments

The authors would like to thank the Associate Editor and the anonymous reviewers for their constructive and helpful comments and suggestions to improve the quality of this paper.

This work was supported in part by the National Natural Science Foundation of China (Nos. 61751205, 61572540 and U1813203) and the Taishan Scholar Project of Shandong Province, China .

Jing Chen received his B.Sc. degree in the School of Mathematical Science and M.Sc. degree in the School of Information Engineering at the Yangzhou University (Yangzhou, China) in 2003 and 2006, respectively, and received his Ph.D. degree in the School of Internet of Things Engineering at the Jiangnan University (Wuxi, China) in 2013. He is currently an associate professor in the School of Science, Jiangnan University (Wuxi, China). He is a Colleges and Universities Blue Project Middle-Aged

References (30)

  • XuH. et al.

    Fitting the exponential autoregressive model through recursive search

    Journal of the Franklin Institute

    (2019)
  • ZhaoD. et al.

    On robust Kalman filter for two-dimensional uncertain linear discrete time-varying systems: A least squares method

    Automatica

    (2019)
  • ZhuQ.M.

    An implicit least squares algorithm for nonlinear rational model parameter estimation

    Applied Mathematical Modelling

    (2005)
  • BillingsS.A. et al.

    Rational model identification using extended least squares algorithm

    International Journal of Control

    (1991)
  • ChenG.Y. et al.

    A regularized variable projection algorithm for separable nonlinear least-squares problems

    IEEE Transactions on Automatic Control

    (2019)
  • Cited by (0)

    Jing Chen received his B.Sc. degree in the School of Mathematical Science and M.Sc. degree in the School of Information Engineering at the Yangzhou University (Yangzhou, China) in 2003 and 2006, respectively, and received his Ph.D. degree in the School of Internet of Things Engineering at the Jiangnan University (Wuxi, China) in 2013. He is currently an associate professor in the School of Science, Jiangnan University (Wuxi, China). He is a Colleges and Universities Blue Project Middle-Aged Academic Leader (Jiangsu, China). His research interests include process control and system identification.

    Biao Huang obtained his Ph.D. degree in Process Control from the University of Alberta, Canada, in 1997. He also had M.Sc. degree (1986) and B.Sc. degree (1983) in Automatic Control from the Beijing University of Aeronautics and Astronautics. Biao Huang joined the University of Alberta in 1997 as an Assistant Professor in the Department of Chemical and Materials Engineering, and is currently a full Professor, NSERC Industrial Research Chair in Control of Oil Sands Processes, and AITF Industry Chair in Process Control (2013–2018). He is a Fellow of the Canadian Academy of Engineering and Fellow of Chemical Institute of Canada. He is recipient of Germany’s Alexander von Humboldt Research Fellowship, Canadian Chemical Engineer Society’s Syncrude Canada Innovation and D.G. Fisher awards, APEGAs Summit Research Excellence award, University of Alberta’s McCalla and Killam Professorship awards, Petro-Canada Young Innovator Award, AsTech Outstanding Achievement in Science & Engineering Award and a Best Paper Award from Journal of Process Control. Biao Huang’s research interests include: data analytics, process control, system identification, control performance assessment, Bayesian methods and state estimation. Biao Huang has applied his expertise extensively in industrial practice.

    Min Gan received the B. S. degree in Computer Science and Engineering from Hubei University of Technology, Wuhan, China, in 2004, and the Ph.D. degree in Control Science and Engineering from Central South University, Changsha, China, in 2010. He is currently a Professor in the College of Computer Science & Technology, Qingdao University, Qingdao, China. His current research interests include statistical learning, system identification and nonlinear time series analysis.

    C.L. Philip Chen is the Chair Professor and Dean of the College of Computer Science and Engineering, South China University of Technology. Being a Program Evaluator of the Accreditation Board of Engineering and Technology Education (ABET) in the U.S., for computer engineering, electrical engineering, and software engineering programs, he successfully architects the University of Macau’s Engineering and Computer Science programs receiving accreditations from Washington/Seoul Accord through Hong Kong Institute of Engineers (HKIE), of which is considered as his utmost contribution in engineering/computer science education for Macau as the former Dean of the Faculty of Science and Technology. He is a Fellow of IEEE, AAAS, IAPR, CAA, and HKIE; a member of Academia Europaea (AE), European Academy of Sciences and Arts (EASA), and International Academy of Systems and Cybernetics Science (IASCYS). He received IEEE Norbert Wiener Award in 2018 for his contribution in systems and cybernetics, and machine learnings. He is also a highly cited researcher by Clarivate Analytics in 2018–2020.

    This work was supported in part by the National Natural Science Foundation of China (Nos. 62073082, 61973137, 61751202 and U1801262), Natural Science and Engineering Research Council of Canada RGPIN-2017-03833, and the National Key Research and Development Program of China (Nos. 2019YFA0706200, 2019YFB1703600). The material in this paper was not presented at any conference. This paper was recommended for publication in revised form by Associate Editor Er-Wei Bai under the direction of Editor Torsten Söderström.

    View full text