An interactive maximum likelihood estimation method for multivariable Hammerstein systems☆
Introduction
The block-oriented nonlinear structures flexibly combine nonlinear static parts and linear dynamic parts, and can be represented by various nonlinear functions and linear functions [1], [2], [3], [4]. Thus they can be widely used in modeling various practical nonlinear systems [4], [5], [6]. From the view of the input-output dimensions, block-oriented systems can be divided into: the single-input and single-output (SISO) systems, the multiple-input and single-output (MISO) systems, and the multiple-input and multiple-output (MIMO) systems. Take the block-oriented Hammerstein systems as examples, the SISO block-oriented Hammerstein systems naturally can be expressed as a bilinear parameter function, and the MISO block-oriented Hammerstein systems can be expressed as a sum of some bilinear parameter functions (their numbers depending on the input numbers), written aswhere
- I)
the system output is a bilinear function about the parameter vectors and from nonlinear block and linear block, is an information matrix about the input .
- II)
the system output is bilinear function about two sets of parameter vectors and from the nonlinear blocks and the linear block, where is an information matrix about the input u2(t), ⋅⋅⋅, .
For SISO block-oriented systems, the identification methods may be sorted according to different classifications. The basic identification strategies can be classified into the stochastic gradient methods [7], [8], [9], the least squares methods [10], [11], [12], [13], the maximum likelihood methods [14], [15], [16], [17], [18], [19], the evolution optimization methods [20], [21], [22], [23], [24], and the sparse theory based methods [25], [26], [27]. The computational styles can be divided into the recursive estimation methods [28], [29], [30], [31], the iterative estimation methods [32], [33], [34], and the hierarchical (interactive) estimation methods [35], [36], [37], [38]. The parametrization modes can be sorted into the over-parametrization modes [39], [40], [41], [42], the key term separation modes [43], [44], [45], [46], and the naturally formed bilinear models [26], [47], [48]. The assistant techniques include the filtering technique [49], [50], [51], [52], the multi-innovation skill [53], [54] and the auxiliary model idea [55], [56], [57].
For multivariable block-oriented systems, due to existing more coupled parameters, it is not easy to extend the identification method for SISO systems to MISO systems and MIMO systems, especially to MIMO systems. For MISO Hammerstein systems, by using the Taylor expansion on a least squares quadratic criterion function, Wang and Zhang investigated an improved least squares algorithm to identify the parameters of the multivariable Hammerstein OEMA system [58].
For MIMO Hammerstein systems with matrix coefficients, based on the over-parametrization model, Salhi and Kamoun studied a recursive least squares estimation method [42], and Salimifard et al. explored gradient and least squares iterative learning methods [59]. For MIMO Hammerstein systems with different types of coefficients: matrix coefficient and scalar coefficients, by combining the hierarchical identification principle with the over-parametrization model, Jafari et al. successfully implemented parameter estimation of the system [40]. By reframing the system into two model forms and by using the hierarchical identification principle, Wang investigated a hierarchical extended least squares algorithm to these two models to alternatively estimate the parameters of the nonlinear parts and the linear part [36].
The motivation of this paper is to explore an interactive maximum likelihood (IML) estimation method to interactively maximize the the logarithmic likelihood function over multiple parameter vectors for a more general model, respectively, not limited on an autoregression model [60], [61]. The characteristic of the IML estimation method is that its identification model contains the minimum number of the unknown parameters, having a high computational efficiency. The contributions of this paper lie in the following aspects.
- •
This paper reframes a multivariable Hammerstein OEMA system into a sum of some bilinear functions about the parameter vectors of the nonlinear parts and the linear part, in which two sets of parameter vectors couple through some known information matrices.
- •
The IML method can directly exerts the maximum likelihood estimation method on the non-regression identification model of the multivariable Hammerstein OEMA system, by maximizing the the logarithmic likelihood function over multiple parameter vectors in a more general model, respectively, without limiting on the autoregressive model form with one parameter vector.
- •
The IML method contains the minimum number of the unknown parameters, and is more efficient than the traditional over-parameterization based identification method, which contains parameter products from the nonlinear blocks and the linear block, resulting in many redundant parameters and very large computational load.
The rest of this paper is organized as follows. Section 2 constructs the identification model for a multivariable Hammerstein CARMA system; Section 3 investigates the interactive maximum likelihood (IML) method for the multivariable Hammerstein CARMA system, including: the description of the IML algorithm, the L step, and the M step; Section 4 gives the computation complexity analysis of the investigated algorithm. Numerical simulations are carried out in Section 5 to demonstrate the effectiveness of the explored algorithm. Finally, some concluding remarks are offered in Section 6.
Section snippets
The bilinear expression of the multivariable Hammerstein system
Consider a multivariable Hammerstein controlled autoregressive moving average system (CARMA) system in Fig. 1,
where is the system input, y(t) is the scalar system output, x2(t), ⋅⋅⋅, xr(t)]T is the internal vector, is stochastic white noise with zero mean; The input nonlinearity gi is modeled as a linear combination of basis functions gik:
The description of the IML algorithm
The maximum likelihood estimation method is to seek the estimated parameters to make the sample is most likely to be observed, i.e., maximizing the probability density function containing the observed sample. The interactive maximum likelihood algorithm aims to get estimates of all parameter vectors by interactively maximizing the logarithmic likelihood function over different parameter vectors, respectively. The logarithmic likelihood function (L-function) is the logarithm of the probability
Analysis of computation complexity
The multiplication/division and addition/subtraction numbers (flops) of the IML algorithm are shown in Table 1 for each iteration step. The total flops is mainly decided by the key term 6rnmN in the IML algorithm, which is smaller than that of the recasted models-based least square iterative algorithm, i.e., referring to [36], [62], [63]. The IML algorithm interactively estimate and the dimensions of each corresponding covariance matrix from different
Example 1
Consider the following multivariable Hammerstein CARMA system with two inputs,The input {u(t)} is taken as an uncorrelated persistently excited signal vector sequence with zero mean and unit variance, and
Conclusions
In this paper, we investigate an interactive maximum likelihood (IML) estimation method for a multivariable Hammerstein CARMA system by maximizing logarithmic likelihood function over multiple parameter vectors in a general model, respectively, and to interactively obtain estimates of these parameter vectors. During the iterative process of the IML estimation algorithm: at each iterations, in the L step, due to existing unknown variables in in the L-function, we need to compute L
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationshipsthat could have appeared to influence the work reported in this paper.
References (65)
- et al.
An innovative fractional order LMS algorithm for power signal parameter estimation
Applied Mathematical Modelling
(2020) - et al.
Identification for Hammerstein nonlinear ARMAX systems based on multi-innovation fractional order stochastic gradient
Signal Processing
(2018) - et al.
Recasted models based hierarchical extended stochastic gradient method for MIMO nonlinear systems
IET Control Theory and Applications
(2017) - et al.
A least-squares parameter estimation algorithm for switched Hammerstein systems with applications to the VOR
IEEE Transactions on Biomedical Engineering
(2005) - et al.
Maximum likelihood identification of dual-rate Hammerstein output error moving average system
IET Control Theory Applications
(2020) - et al.
Global convergence of the EM algorithm for ARX models with uncertain communication channels
Systems & Control Letters https
(2020) - et al.
Nonlinear system identification based on ANFIS-Hammerstein model using gravitational search algorithm
Applied Intelligence
(2018) - et al.
Gradient based particle filter algorithm for an ARX model with nonlinear communication output, IEEE transactions on systems
Man and Cybernetics: Systems
(2020) - et al.
Parameter estimation algorithms for Hammerstein time-delay systems based on the OMP scheme
IET Signal Processing
(2017) - et al.
Convergence analysis for recursive Hammerstein identification
Automatica
(2016)
Recursive parameter identification of the dynamical models for bilinear state space systems
Nonlinear Dynamics
Iterative identification of block-oriented nonlinear systems based on biconvex optimization
Systems & Control Letters
Application of the newton iteration algorithm to the parameter estimation for dynamical systems
Journal of Computational and Applied Mathematics
Hierarchical recursive least squares algorithm for Hammerstein systems using the filtering method
Nonlinear Dynamics
Hierarchical stochastic gradient algorithm and its performance analysis for a class of bilinear-in-parameter systems
Circuits, Systems and Signal Processing
A changing forgetting factor RLS for online identification of nonlinear systems based on ELM-Hammerstein model
Neural Computing and Applications
Recursive least squares algorithm for nonlinear dual-rate systems using missing-output estimation model
Circuits, Systems and Signal Processing
Identification of nonlinear dynamic systems with input saturation and output backlash using three-block cascade models
Journal of the Franklin Institute
Identification of nonlinear cascade systems with output hysteresis based on the key term separation principle
Applied Mathematical Modelling
Iterative identification of nonlinear dynamic systems with output backlash using three-block cascade models
Nonlinear Dynamics
Hierarchical least squares identification for feedback nonlinear equation-error systems
Journal of the Franklin Institute
Decoupled parameter estimation methods for Hammerstein systems by using filtering techniques
IEEE Access
A multi-innovation state and parameter estimation algorithm for a state space system with d-step state-delay
Signal Processing
Multi-step-length gradient iterative algorithm for equation-error type models
Systems & Control Letters
Auxiliary model based least squares iterative algorithms for parameter estimation of bilinear systems using interval-varying measurements
IEEE Access
Design of normalized fractional adaptive algorithms for parameter estimation of control autoregressive autoregressive systems
Applied Mathematical Modelling
Normalized fractional adaptive methods for nonlinear control autoregressive systems
Applied Mathematical Modelling
Computational efficiency of the identification methods. Part B: Iterative algorithms
Journal of Nanjing University of Information Science and Technology: Ntural Science Edition
Block-Oriented Nonlinear System Identification
Nonlinear system identification using fractional Hammerstein–Wiener models
Nonlinear Dynamics
Interval variable step-size spline adaptive filter for the identification of nonlinear block-oriented system
Nonlinear Dynamics
Design of sign fractional optimization paradigms for parameter estimation of nonlinear Hammerstein systems
Neural Computing and Applications
Cited by (0)
- ☆
This work was supported by the National Natural Science Foundation of China (No. 61873138).