Identification of Wiener–Hammerstein models based on variational bayesian approach in the presence of process noise

Abstract

This paper addresses the identification of Wiener–Hammerstein (WH) models in the presence of process and measurement noises, which has not been well studied yet in the existing works. To achieve an unbiased estimation, the model parameters are obtained by maximizing the likelihood function, which is solved in the expectation-maximization framework. Due to the difficulty of computing the posterior distributions of the latent variables of WH models, variational Bayes (VB) is used here, and a method for approximating the posterior distributions based on Monte Carlo integral is proposed in VB framework. To the best of our knowledge, it is the first time to use VB approach for WH model identification. Two simulation examples demonstrate the effectiveness of the proposed method. Moreover, the proposed method is used for a WH benchmark problem, and the results show that it improves the identification performance.

Introduction

Nonlinear system identification is generally recognized as a difficult and challenging problem and has attracted much attention in the last decades [1], [2], [3]. Generally speaking, there does not exist a general identification method for all nonlinear models. As a result, the identification methods for specific nonlinear structures have been proposed successively, such as the nonlinear state space models [4], NARMAX (Nonlinear Autoregressive Moving Average with eXogenous inputs) models [5] and block-oriented nonlinear models [6], [7], [8], [9]. Among those models, block-oriented model has been recognized as useful since it is quite simple and easy to be understood. The representative structure of the block-oriented model is Wiener and Hammerstein structure. In this paper, we consider the identification of Wiener–Hammerstein (WH) model, a relatively complex model for block-oriented models.

While many papers have considered the identification of WH models [10], [11], [12], [13], [14], [15], there still exist some problems needing to be further studied. The structure of the WH system is shown in Fig. 1, where $x_t$ is the intermediate, unmeasurable variable and $y_t$ the output variable. In most existing works, the authors only considered the identification in the case that $y_t$ is disturbed by measurement noise $e_t$. In fact, the intermediate variable $x_t$ can also be disturbed by noise $w_t$ (called as process noise). However, few works considered this case, which was addressed by Ljung in Hagenblad et al. [11]. If both the process and measurement noises exist, some existing methods will fail to estimate the model parameters. Among the identification methods for WH models, the method based on the prediction error minimization (PEM) is most widely used [10], [16], [17]. However, since the nonlinearity and the existence of the process noise, the estimation of some parameters will be biased, which has been addressed in Hagenblad et al. [11]. An alternative method to identify the parameter is the Maximization Likelihood estimation (MLE), which can provide an unbiased estimation compared with PEM.

MLE is the widely used method for system identification, and it was first used for Wiener system identification in Hagenblad et al. [11], where the likelihood function was computed directly and the parameters were obtained by maximizing the likelihood function. If we straightforwardly compute the likelihood function, a lot of exponential functions and integrals need to be calculated. It is sometimes infeasible to directly obtain the likelihood function. To solve this problem, an alternative method, Expectation-Maximization (EM) algorithm is effective to maximize the likelihood function and was widely used for system identification [18], [19], [20], [21]. In [13], the authors used EM algorithm to identify Wiener models in the absence of process noise. In EM framework, the posterior probabilities of the latent variables must be evaluated. In fact, for WH model in the presence of process noise, it is difficult to obtain the posterior distributions of the latent variables $x_t$. In these similar cases, Variational Bayes [22], [23] provides a method to compute the posterior distributions of the latent variables and was widely used to extend the applicability of the EM algorithm (called as Variational Bayesian EM, VBEM). Since the advantage of VBEM, it has been used recently for system identification [24], [25], [26]. However, most of these works focused on the linear model identification. The application of the VBEM for nonlinear system identification, especially for WH models has not been well studied.

In this paper, we consider the identification of WH models in the presence of process and measurement noises. MLE is used to estimate the model parameters. To obtain the MLE, the VBEM algorithm is used here. We propose an approximate method based on the Monte Carlo integral to optimize the posterior distributions of the intermediate variables with the unknown distribution form. Two numerical examples show the effectiveness and merits of the proposed method. The proposed method is also used for a WH benchmark problem, and the results show that it outperforms the existing methods.

This paper is organized as follows. In Section 2, we describe the identification problem considered in this paper. In Section 3, the framework of the VBEM is described and the proposed VBEM for WH model identification is detailed. The simulation studies and a benchmark WH identification are given in Sections 4 and 5 respectively. Section 6 addresses the conclusions.