An interactive maximum likelihood estimation method for multivariable Hammerstein systems

https://doi.org/10.1016/j.jfranklin.2020.09.005Get rights and content

Abstract

For a multivariable Hammerstein controlled autoregressive moving average system (CARMA) system, the identification difficulty is hard to parameterize the system into an quasi auto-regression form to which the standard least square method can apply. By using an interactive maximum likelihood (IML) estimation method, this paper interactively maximizes the logarithmic likelihood function over multiple parameter vectors in a more general model, respectively. The details include: (1) reframe the system into a sum of some bilinear functions about the parameter vectors of the nonlinear part and the linear part; (2) interactively maximize the logarithmic likelihood function over each parameter vector to get their estimates; (3) when updating one parameter vector, substitute other parameter vectors or unknown information vectors by their estimates.

The advantage of the IML algorithm is that it overcomes the limit on an autoregressive model form with one parameter vector. The IML method is simple to understand and easy to implement. Numerical simulations indicate that the explored IML algorithm is capable of generating accurate parameter estimates, and easy to implement on-line.

Introduction

The block-oriented nonlinear structures flexibly combine nonlinear static parts and linear dynamic parts, and can be represented by various nonlinear functions and linear functions [1], [2], [3], [4]. Thus they can be widely used in modeling various practical nonlinear systems [4], [5], [6]. From the view of the input-output dimensions, block-oriented systems can be divided into: the single-input and single-output (SISO) systems, the multiple-input and single-output (MISO) systems, and the multiple-input and multiple-output (MIMO) systems. Take the block-oriented Hammerstein systems as examples, the SISO block-oriented Hammerstein systems naturally can be expressed as a bilinear parameter function, and the MISO block-oriented Hammerstein systems can be expressed as a sum of some bilinear parameter functions (their numbers depending on the input numbers), written asy(t)|SISO=βTf[u(t)]γ,y(t)|MISO=i=1rβiTFi[ui(t)]γi,where

  • I)

    the system output y(t)|SISOR is a bilinear function about the parameter vectors βRn and γRm from nonlinear block and linear block, f[u(t)]Rn×m is an information matrix about the input u(t)R.

  • II)

    the system output y(t)|MISOR is bilinear function about two sets of parameter vectors βiRn and γiRm from the nonlinear blocks and the linear block, where Fi[ui(t)]Rn×m is an information matrix about the input u(t)=[u1(t), u2(t), ⋅⋅⋅, ur(t)]TRr.

For SISO block-oriented systems, the identification methods may be sorted according to different classifications. The basic identification strategies can be classified into the stochastic gradient methods [7], [8], [9], the least squares methods [10], [11], [12], [13], the maximum likelihood methods [14], [15], [16], [17], [18], [19], the evolution optimization methods [20], [21], [22], [23], [24], and the sparse theory based methods [25], [26], [27]. The computational styles can be divided into the recursive estimation methods [28], [29], [30], [31], the iterative estimation methods [32], [33], [34], and the hierarchical (interactive) estimation methods [35], [36], [37], [38]. The parametrization modes can be sorted into the over-parametrization modes [39], [40], [41], [42], the key term separation modes [43], [44], [45], [46], and the naturally formed bilinear models [26], [47], [48]. The assistant techniques include the filtering technique [49], [50], [51], [52], the multi-innovation skill [53], [54] and the auxiliary model idea [55], [56], [57].

For multivariable block-oriented systems, due to existing more coupled parameters, it is not easy to extend the identification method for SISO systems to MISO systems and MIMO systems, especially to MIMO systems. For MISO Hammerstein systems, by using the Taylor expansion on a least squares quadratic criterion function, Wang and Zhang investigated an improved least squares algorithm to identify the parameters of the multivariable Hammerstein OEMA system [58].

For MIMO Hammerstein systems with matrix coefficients, based on the over-parametrization model, Salhi and Kamoun studied a recursive least squares estimation method [42], and Salimifard et al. explored gradient and least squares iterative learning methods [59]. For MIMO Hammerstein systems with different types of coefficients: matrix coefficient and scalar coefficients, by combining the hierarchical identification principle with the over-parametrization model, Jafari et al. successfully implemented parameter estimation of the system [40]. By reframing the system into two model forms and by using the hierarchical identification principle, Wang investigated a hierarchical extended least squares algorithm to these two models to alternatively estimate the parameters of the nonlinear parts and the linear part [36].

The motivation of this paper is to explore an interactive maximum likelihood (IML) estimation method to interactively maximize the the logarithmic likelihood function over multiple parameter vectors for a more general model, respectively, not limited on an autoregression model [60], [61]. The characteristic of the IML estimation method is that its identification model contains the minimum number of the unknown parameters, having a high computational efficiency. The contributions of this paper lie in the following aspects.

  • This paper reframes a multivariable Hammerstein OEMA system into a sum of some bilinear functions about the parameter vectors of the nonlinear parts and the linear part, in which two sets of parameter vectors couple through some known information matrices.

  • The IML method can directly exerts the maximum likelihood estimation method on the non-regression identification model of the multivariable Hammerstein OEMA system, by maximizing the the logarithmic likelihood function over multiple parameter vectors in a more general model, respectively, without limiting on the autoregressive model form with one parameter vector.

  • The IML method contains the minimum number of the unknown parameters, and is more efficient than the traditional over-parameterization based identification method, which contains parameter products from the nonlinear blocks and the linear block, resulting in many redundant parameters and very large computational load.

The rest of this paper is organized as follows. Section 2 constructs the identification model for a multivariable Hammerstein CARMA system; Section 3 investigates the interactive maximum likelihood (IML) method for the multivariable Hammerstein CARMA system, including: the description of the IML algorithm, the L step, and the M step; Section 4 gives the computation complexity analysis of the investigated algorithm. Numerical simulations are carried out in Section 5 to demonstrate the effectiveness of the explored algorithm. Finally, some concluding remarks are offered in Section 6.

Section snippets

The bilinear expression of the multivariable Hammerstein system

Consider a multivariable Hammerstein controlled autoregressive moving average system (CARMA) system in Fig. 1,

x(t)=[x1(t)x2(t)xr(t)],xi(t)=gi[ui(t)],i=1,2,,r,α(z)y(t)=β(z)x(t)+λ(z)v(t),where u(t)=[u1(t),u2(t),,ur(t)]TRr is the system input, y(t)R is the scalar system output, x(t)=[x1(t), x2(t), ⋅⋅⋅, xr(t)]T Rr is the internal vector, v(t)R is stochastic white noise with zero mean; The input nonlinearity gi is modeled as a linear combination of basis functions gik:xi(t)=gi[ui(t)]=k=1mγikg

The description of the IML algorithm

The maximum likelihood estimation method is to seek the estimated parameters to make the sample is most likely to be observed, i.e., maximizing the probability density function containing the observed sample. The interactive maximum likelihood algorithm aims to get estimates of all parameter vectors by interactively maximizing the logarithmic likelihood function over different parameter vectors, respectively. The logarithmic likelihood function (L-function) is the logarithm of the probability

Analysis of computation complexity

The multiplication/division and addition/subtraction numbers (flops) of the IML algorithm are shown in Table 1 for each iteration step. The total flops is mainly decided by the key term 6rnmN in the IML algorithm, which is smaller than that of the recasted models-based least square iterative algorithm, i.e., 2(rmn+2n)2N, referring to [36], [62], [63]. The IML algorithm interactively estimate ϑ(k+1), βi(k+1) and γi(k+1), the dimensions of each corresponding covariance matrix from different

Example 1

Consider the following multivariable Hammerstein CARMA system with two inputs,y(t)=β(z)α(z)x(t)+λ(z)v(t),x(t)=[x1(t)x2(t)]=[0.09u1(t)+0.11u12(t)0.24u2(t)+0.40u22(t)],α(z)=11.50z1+1.10z2,λ(z)=0.34z10.30z2,β(z)=[2.60z10.75z2,1.80z1+1.80z2],α=[1.501.10],λ=[0.340.30],ϑ=[αλ],β1=[2.600.75],β2=[1.801.80],β=[β1β2],γ1=[0.090.11],γ2=[0.240.40],γ=[γ1γ2].The input {u(t)} is taken as an uncorrelated persistently excited signal vector sequence with zero mean and unit variance, and {v(t)}

Conclusions

In this paper, we investigate an interactive maximum likelihood (IML) estimation method for a multivariable Hammerstein CARMA system by maximizing logarithmic likelihood function over multiple parameter vectors in a general model, respectively, and to interactively obtain estimates of these parameter vectors. During the iterative process of the IML estimation algorithm: at each k+1 iterations, in the L step, due to existing unknown variables v(t) in ψ(t) in the L-function, we need to compute L

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationshipsthat could have appeared to influence the work reported in this paper.

References (65)

  • X. Zhang et al.

    Recursive parameter identification of the dynamical models for bilinear state space systems

    Nonlinear Dynamics

    (2017)
  • G.Q. Li et al.

    Iterative identification of block-oriented nonlinear systems based on biconvex optimization

    Systems & Control Letters

    (2015)
  • L. Xu

    Application of the newton iteration algorithm to the parameter estimation for dynamical systems

    Journal of Computational and Applied Mathematics

    (2015)
  • Z.Y. Wang et al.

    Hierarchical recursive least squares algorithm for Hammerstein systems using the filtering method

    Nonlinear Dynamics

    (2015)
  • F. Ding et al.

    Hierarchical stochastic gradient algorithm and its performance analysis for a class of bilinear-in-parameter systems

    Circuits, Systems and Signal Processing

    (2017)
  • Y.G. Tang et al.

    A changing forgetting factor RLS for online identification of nonlinear systems based on ELM-Hammerstein model

    Neural Computing and Applications

    (2017)
  • J. Chen et al.

    Recursive least squares algorithm for nonlinear dual-rate systems using missing-output estimation model

    Circuits, Systems and Signal Processing

    (2017)
  • J. Voros

    Identification of nonlinear dynamic systems with input saturation and output backlash using three-block cascade models

    Journal of the Franklin Institute

    (2014)
  • J. Vörös

    Identification of nonlinear cascade systems with output hysteresis based on the key term separation principle

    Applied Mathematical Modelling

    (2015)
  • J. Vörös

    Iterative identification of nonlinear dynamic systems with output backlash using three-block cascade models

    Nonlinear Dynamics

    (2015)
  • F. Ding et al.

    Hierarchical least squares identification for feedback nonlinear equation-error systems

    Journal of the Franklin Institute

    (2020)
  • D.Q. Wang et al.

    Decoupled parameter estimation methods for Hammerstein systems by using filtering techniques

    IEEE Access

    (2018)
  • L. Xu et al.

    A multi-innovation state and parameter estimation algorithm for a state space system with d-step state-delay

    Signal Processing

    (2017)
  • J. Chen et al.

    Multi-step-length gradient iterative algorithm for equation-error type models

    Systems & Control Letters

    (2018)
  • M.H. Li et al.

    Auxiliary model based least squares iterative algorithms for parameter estimation of bilinear systems using interval-varying measurements

    IEEE Access

    (2018)
  • N.I. Chaudhary et al.

    Design of normalized fractional adaptive algorithms for parameter estimation of control autoregressive autoregressive systems

    Applied Mathematical Modelling

    (2018)
  • N.I. Chaudhary et al.

    Normalized fractional adaptive methods for nonlinear control autoregressive systems

    Applied Mathematical Modelling

    (2019)
  • F. Ding

    Computational efficiency of the identification methods. Part B: Iterative algorithms

    Journal of Nanjing University of Information Science and Technology: Ntural Science Edition

    (2012)
  • F. Giri et al.

    Block-Oriented Nonlinear System Identification

    (2010)
  • K. Hammar et al.

    Nonlinear system identification using fractional Hammerstein–Wiener models

    Nonlinear Dynamics

    (2019)
  • L.D. Yang et al.

    Interval variable step-size spline adaptive filter for the identification of nonlinear block-oriented system

    Nonlinear Dynamics

    (2019)
  • N.I. Chaudhary et al.

    Design of sign fractional optimization paradigms for parameter estimation of nonlinear Hammerstein systems

    Neural Computing and Applications

    (2020)
  • Cited by (0)

    This work was supported by the National Natural Science Foundation of China (No. 61873138).

    View full text