Elsevier

Signal Processing

Volume 190, January 2022, 108314
Signal Processing

A Gaussian mixture regression model based adaptive filter for non-Gaussian noise without a priori statistic

https://doi.org/10.1016/j.sigpro.2021.108314Get rights and content

Highlights

  • A new variational Bayesian Gaussian mixture filter is proposed to estimate states under unknown non-Gaussian measurement noises.

Abstract

In many engineering systems, the distribution of measurement noise is unknown and non-Gaussian, such as skewed, multimodal, time-varying distributions and we can not obtain these prior statistics in advance. How to achieve state estimation via this kind of non-Gaussian measurement noises without prior statistic information is a challenging problem. In this paper, a novel Gaussian mixture regression model (GMRM) is proposed to model the unknown non-Gaussian measurement likelihood for Bayesian update to achieve nonlinear state estimation. Without any prior assumption or limitation of measurement noises’ statistics and distributions, the GMRM can still describe the measurement likelihood accurately based on a group of parameters which are adjusted by maximizing the evidence lower bound. Based on the optimized GMRM, a new variational Bayesian Gaussian mixture filter is proposed by using the variational Bayesian approach. To eliminate the influence of the initialization of the introduced parameters, a learning scheme is proposed to adaptively optimize their hyper-parameters based on the historical measurements. Finally, simulation examples are employed to illustrate the effectiveness of the filter.

Introduction

State estimation for dynamic systems is one of the most important subfields of automation and control. The main purpose of state estimation is to use the received noisy measurements to calculate the posterior probability density function (PDF) of system states. In linear Gaussian state and measurement systems, given the distribution and its parameters of noises, Kalman filter (KF) is an optimal state estimator in the sense of minimum variance [1]. However, in most practical cases, state-space models are usually nonlinear. The distributions, statistics and their parameters of measurement noises are usually non-Gaussian and unknown. For example, many soft sensing approaches of industrial processes, are inherently non-Gaussian due to complicated mechanisms and multioperation conditions/phases [2]. In global positioning systems and integrated navigation systems [3], environment around sensors will change inevitably and unpredictably so that the statistic properties of measurement noises are time-varying and non-Gaussian. In fault diagnosis and tolerant control systems, inputs are of the non-Gaussian type [4]. Even for Gaussian inputs, nonlinearities in stochastic systems may lead to non-Gaussian outputs [4].

Hence, in different existing filters, different prior assumptions of distributions or statistics are made to different kinds of unknown measurement noises for achieving state estimation. In other words, when their assumptions hold, actually the distributions and statistics of measurement noises are known and only the parameters in the assumed distribution are unknown, which need to be calculated.

  • 1.

    In many existing adaptive Kalman filters (AKFs), such as maximum likelihood [5], covariance matching [6], correlation [7], variational Bayesian (VB) methods [8], [9], [10], [11], it is assumed that measurement noises obey a Gaussian distribution. This means that the Gaussian distribution of the noise is known and only its covariance is unknown. However, the Gaussian assumption in non-Gaussian systems will greatly influence the state estimation accuracy.

  • 2.

    In some existing non-Gaussian filters, measurement noises are assumed as a heavy-tailed distribution, for which the Student-t distribution is useful [12]. In nonparametric methods, many Student-t weighted integral rules have been proposed to implement nonlinear Student-t based KFs and smoothers, such as sigma-point methods of degrees 3 and 5 [13] and stochastic methods [14]. As for parametric approaches, the non-Gaussian measurement noise is modeled as a Student-t distribution [15] or a weighted sum form of Gaussian and a Student-t distributions [1]. Then, the system state and the parameters of the assumed distribution are estimated via VB approach. In addition, the Laplace distribution is also popular to model heavy-tailed noises in KF to achieve state estimation [16] and target detection [17].

  • 3.

    Besides the assumptions of Gaussian and heavy-tailed distributions, skewed distributions of noises are also assumed. Skew Student-t statistics are used widely in solving the problem of non-Gaussian noises to estimate states. In [18], the extended KF is combined with a skew Student t-distribution to estimate the state and identify parameters of measurement noises by the VB approach. The skew Student t-distribution has also been applied within sigma-point based filters in the wireless localization problem [19]. In [20], [21], heavy-tailed and skewed features are both considered.

Moreover, the particle filter (PF) [22] can also estimate states with non-Gaussian noises, but its computational burden is heavy. For decreasing the computational cost of PF, an adaptive marginalized particle filter (MPF) is further developed, which can speed up the calculation of nonlinear integrals [23], [24]. MPF can also be combined with the VB approach [25], in which the parametric Student-t distribution is inferred by the VB approach and states are estimated using MPF via sufficient statistics. Moreover, the interaction multiple model (IMM) is also a useful method, which operates multiple KFs corresponding to different measurement noise models [26]. However, a set of measurement noise models needs to be given in advance, which limits its application. Further, a modified VB-IMM filter [27] is proposed, in which system states together with the noise parameters for each model are inferred by the VB approach.

In the above mentioned methods, although they work well under their prior assumptions of measurement noises’ distributions and statistics, there exist two shortcomings.

  • 1.

    Too strong dependency of prior assumptions of measurement noises. The pre-condition for the existing methods to work well is to assume correct prior statistics and distributions of non-Gaussian measurement noises, such as heavy-tailed or skewed distributions. The use of wrong prior statistics can result in substantial estimation errors or even filtering divergence. For example, in Xu et al. [25], if measurement noises do not obey a heavy-tailed distribution, a Student-t distribution-based model will greatly degrade the estimation performance. In IMM [27], when the prior model set does not contain the true measurement noise, the estimation accuracy will also be influenced badly. However, in many practical cases, it is hard to obtain accurate prior statistics of measurement noises in advance. Consequently, correct prior assumptions of measurement noises in these methods are hard to be hold and their estimation performance can not be guaranteed accordingly.

  • 2.

    Too sensitive to the values of hyper-parameters. In the above mentioned VB-based filters, the introduction of hyper-parameters [1], [23], [24] is inevitable, which need to be artificially set in advance. However, there are not reasonable rules or schemes to adaptively set these hyper-parameters. If the hyper-parameters setting deviates from the true situation, the estimation performance will be influenced greatly, which will further limit filters’ application range.

The aim of this paper is to achieve nonlinear state estimation without any prior assumptions of unknown non-Gaussian measurement noises’ distributions and statistics. To this end, there are two challenges we need to overcome.

  • 1.

    Without prior assumptions of measurement noises, how to accurately model the true measurement likelihood probability density function (ML-PDF) in implementing the Bayesian update to estimate the state. Due to the lack of prior information of non-Gaussian measurement noises, the modeling of ML-PDF will become intractable, which hinders the Bayesian update. To proceed with the Bayesian update, a parametric optimizable model is necessary for modeling the true ML-PDF, which does not rely on prior assumptions.

  • 2.

    How to adaptively calculate the values of hyper-parameters, rather than artificial setting. With different measurement noises, proper setting of hyper-parameters is also different. Unreasonably artificial setting will deviate the optimizable model from the true ML-PDF, which further causes poor estimation accuracy. Hence, an optimization scheme is necessary to adaptively calculate hyper-parameters’ values to adapt the algorithm to different non-Gaussian measurement noises.

In view of above challenges, a novel parametric Gaussian mixture regression model (GMRM) is proposed to model the unknown and non-Gaussian ML-PDF. In GMRM, a set of adjustable parameters, variational parameters (VPs) and mixture coefficient (MC), is introduced to adjust the distribution and statistic of every component and their mixing probabilities in GMRM. Then, based on the GMRM, a variational Bayesian Gaussian mixture filter (VBGMF) is proposed, in which the state, MC and VPs are inferred jointly employing the VB approach. This paper’s contributions consist in the following:

  • 1.

    Breaking the dependence of prior information of measurement noises. Different from the traditional Gaussian mixture model, each component of GMRM essentially fits states and measurements as a parametric Gaussian regression process with VPs. The evidence lower bound (ELBO) in the VB optimization framework is the objective function with respective to the unknown parameters’ distributions [28]. Hence, through maximizing the ELBO to optimize the introduced VPs and MC based on measurements in a window length (WL), the mean and covariance of each component and its weight of mixture in GMRM can both be adaptively adjusted. As a result, without any prior statistics of measurement noises, the proposed GMRM can still accurately model ML-PDF and accommodate different unknown and non-Gaussian noises.

  • 2.

    Avoiding the unreasonably artificial setting of hyper-parameters. A new scheme is proposed to adaptively optimize the introduced hyper-parameters by learning the information of previous measurements, rather than artificial setting. For different non-Gaussian measurement noises, the estimation performance is still stable because hyper-parameters can be initialized reasonably according to the knowledge learned from previous measurements.

The rest of this paper is organized as follows: the problem and modeling process are formulated in Section 2. In Section 3, the variational iterative process to infer states, VPs and MC is derived and VBGMF is proposed. The adaptive optimization algorithm of the introduced hyper-parameters is proposed in Section 4. Section 5 shows the complete algorithm flow of VBGMF. Section 6 gives the simulation results illustrating that our proposed VBGMF is effective. Concluding remarks are given in Section 7.

We will use the following notations. The superscripts “1” and “T” represent the matrix inverse and transpose operations, respectively; |·| and Tr(·) denote the matrix determinant and trace, respectively; N(x|μ,P) denotes that variable x obeys Gaussian distribution with mean μ and covariance P; p(x) denotes the expectation operator with respect to the distribution p(x); p(x|z) represents the density of x conditioned on z; I denotes the unit matrix; Γ(·) denotes the gamma function. The superscripts “” and “”, used as the hat of random variables, represent the estimate and the estimation error, respectively. For example, x^ denotes the estimate of variable x and its estimation error is x˜=xx^. Define operators S(x)=xxT and D(x,P)=xTPx.

Section snippets

Problem formulation and modeling process of measurement likelihood

Consider the nonlinear model, i.e.,xk=f(xk1)+wk1zk=h(xk)+vkwhere k denotes the sampling time, xkRn and zkRm denote the state and measurement, respectively. f(·) and h(·) are the nonlinear state and measurement functions, respectively. The system noise wk obeys the zero-mean Gaussian distribution, i.e., wkN(wk|0,Q).

Note that there are not limitation and prior assumptions about the expression form or distribution of the measurement noise vk. In other words, the measurement noise vk could be

Variational Bayesian Gaussian mixture adaptive filter

In this section, we will discuss that how to derive the variational iteration to jointly estimate system states and optimize the VPs A, B, latent variable Yk and MC π. To this end, the core is to express the joint posteriori PDF p(xk,A,B,Yk,π|Z1k). Because there is no analytical solution in the nonlinear system (2), VB approach is therefore employed to obtain a suboptimal approximation for the joint posterior PDF. Based on the VB approach, we are going to look for an approximate solution by

Optimization of hyper-parameters

After the variational iteration process, including EC, EF and M-steps, is derived to iteratively infer states, VPs and MC in the previous section, there still exists one challenge, i.e., how to adaptively optimize the hyper-parameters in (7) and (8) at the beginning of the variational iteration. In this paper, the main idea is to raise a new lower bound at the beginning of variational iteration t=0, which is considered as an objective function with respect to the hyper-parameters, i.e.,Lt=0(q)=

Algorithm flow

The proposed VBGMF is composed of hyper-parameter optimization in the HP-E and M steps, GMRM optimization in the EC and EF steps, posterior state estimation in the M step. In this section, the details of the proposed VBGMF are shown in the following pseudocode, where Nmax is the maximum number of variational iterations and K is the length of sampling time.

The terminal condition of the variational iteration is to measure the difference between the t1 and tth variational state posteriori PDFs qt

Simulation

In this section, our proposed VBGMF will be compared with VB-extended KF (VB-EKF), VB-unscented KF (VB-UKF), VB-cubature KF (VB-CKF), MPF-VBS and IMM in two different measurement noise environments. In order to evaluate the performance of different filters, we use the root mean square error (RMSE) over Nmc=100 simulation runs:RMSEk[i]=1Nmcr=1Nmc(xkr[i]x^kr[i])2,k=1,2,,K where K is the simulation length of sampling time; xkr[i] and x^kr[i] are the true and estimated values of the i scalar

Conclusion

In this paper, a new algorithm called VBGMF was developed for dealing with the nonlinear filtering problem in unknown and non-Gaussian noise environment without a priori. Through designing GMRM to fit ML-PDF, VBGMF achieves state estimation and does not rely on any prior information of measurement noises. ML-PDFs of different sampling times share a same set of VPs and MC. Hence, the optimization of VPs and MC also depends on all measurements in a WL, not only current one, which improves the

Declaration of Competing Interest

Authors declare that they have no conflict of interest.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 61873208, 61573287, 61203234, 61135001, and 61374023, in part by the Shaanxi Natural Science Foundation of China under Grant 2017JM6006, in part by the Aviation Science Foundation of China under Grant 2016ZC53018, in part by the Fundamental Research Funds for Central Universities under Grant 3102017jghk02009, in part by the National Key Research and Development Program of China under Grant

References (34)

  • M. Oussalah et al.

    Adaptive Kalman filter for noise identification

    Proceedings of the International Seminar on Modal Analysis

    (2001)
  • S. Särkkä et al.

    Recursive noise adaptive Kalman filtering by variational Bayesian approximations

    IEEE Trans. Autom. Control

    (2009)
  • P. Dong et al.

    Variational Bayesian adaptive cubature information filter based on Wishart distribution

    IEEE Trans. Autom. Control

    (2017)
  • Y. Huang et al.

    A novel adaptive Kalman filter with inaccurate process and measurement noise covariance matrices

    IEEE Trans. Autom. Control

    (2018)
  • F. Tronarp et al.

    Sigma-point filtering for nonlinear systems with non-additive heavy-tailed noise

    2016 19th International Conference on Information Fusion (FUSION)

    (2016)
  • O. Straka et al.

    Stochastic integration Student’s-t filter

    2017 20th International Conference on Information Fusion (Fusion)

    (2017)
  • J. Li et al.

    A fast distributed variational Bayesian filtering for multisensor LTV system with non-Gaussian noise

    IEEE Trans. Cybern.

    (2019)
  • Cited by (8)

    • A Gaussian-multivariate Laplacian mixture distribution based robust cubature Kalman filter

      2023, Measurement: Journal of the International Measurement Confederation
    • An improved adaptive robust information filter for spacecraft relative navigation

      2022, Aerospace Science and Technology
      Citation Excerpt :

      Among all the adaptive filters, the VB-based adaptive filters have always been the focus of research. Based on VB theory, there are several adaptive KFs developed, such as VB adaptive CIF with Wishart distribution [18], Gaussian process regression-aided VB adaptive UKF [19], and VB adaptive Gaussian mixture filter [20]. The robust filters can suppress outliers to improve estimation accuracy.

    • An adaptive cubature Kalman filter for nonlinear systems against randomly occurring injection attacks

      2022, Applied Mathematics and Computation
      Citation Excerpt :

      In order to overcome the drawback of the above methods, a new methodology, the so-called variational Bayesian (VB), is proposed to approximate the posterior distribution, which has the advantages of low computational cost and analytic tractability [22,23]. Over the past few years, a series of VB-based adaptive filters have been proposed to solve the state estimation problems with unknown noise statistics and other problems [24–29]. To estimate the dynamic state and the time-varying measurement noise parameters, a VB-based AKF is proposed in [23], where the unknown noise covariance is modeled by inverse-Gamma distribution.

    View all citing articles on Scopus
    View full text