Model and data reduction for data assimilation: Particle filters employing projected forecasts and data with application to a shallow water model

https://doi.org/10.1016/j.camwa.2021.05.026Get rights and content

Abstract

The understanding of nonlinear, high dimensional flows, e.g, atmospheric and ocean flows, is critical to address the impacts of global climate change. Data Assimilation (DA) techniques combine physical models and observational data, often in a Bayesian framework, to predict the future state of the model and the uncertainty in this prediction. Inherent in these systems are noise (Gaussian and non-Gaussian), nonlinearity, and high dimensionality that pose challenges to making accurate predictions. To address these issues we investigate the use of both model and data dimension reduction based on techniques including Assimilation in the Unstable Subspace (AUS), Proper Orthogonal Decomposition (POD), and Dynamic Mode Decomposition (DMD). Algorithms to take advantage of projected physical and data models may be combined with DA techniques such as Ensemble Kalman Filter (EnKF) and Particle Filter (PF) variants. The projected DA techniques are developed for the optimal proposal particle filter and applied to the Lorenz'96 model (L96) and Shallow Water Equations (SWE) to test the efficacy of our techniques in high dimensional, nonlinear systems.

Introduction

Several important challenges impede the development of Data Assimilation (DA) techniques: nonlinear physical models, high dimensional models and data, and non-Gaussian posterior distributions. Particle filters and their variants are well suited to handle nonlinearities and non-Gaussian distributions, but due to so-called filter degeneracy still struggle to make accurate predictions for high dimensional problems, especially partial differential equations (PDEs). Variants of particle filters have been developed to ameliorate this difficulty, including implicit particle filters, proposal density methods, the optimal proposal, [1], [2], [3], [4]. A related set of works analyze the performance of particle filters: [5], [6] show that particle filter degeneracy is induced by a ‘curse of dimensionality’ associated with the data and/or model dimension.

Our contribution in this paper is to develop an approach to particle filtering based on reduced-dimension physical and data models, employing projections of the data and model spaces. The crucial benefit of this approach is that it directly targets the filter degeneracy induced by, for example, simulating some high-dimensional PDEs, while maintaining the Bayesian framework of the particle filter suitable for nonlinear, non-Gaussian DA. We build on the substantial development of Assimilation in the Unstable Subspace (AUS) methods (see, e.g., [7], [8], [9], [10], [11]), and recent work on projected data models for particle filtering [12]. The AUS methods are based on state space projections to subspaces spanned by Lyapunov vectors corresponding to the dominant Lyapunov exponents of the system dynamics. The AUS state space projections can greatly improve Kalman Filter (KF) and variational DA schemes: for example, an AUS implementation of 3D-Var works efficiently in high dimensions [13]. However for ensemble-based DA schemes, including the particle filter, the AUS approach has limited promise as the ensemble forecast already tends to align with the dominant Lyapunov exponents [14]. Here we extend the AUS approach by developing projections based on other common model reduction techniques, such as Proper Orthogonal Decomposition (POD) and Dynamic Mode Decomposition (DMD), and by considering combinations of projected physical and data models. Although our focus here is on particle filters (both the standard and optimal proposal forms), the projected models developed here are also applicable to KF and variational techniques. We combine dimension reduction in both the physical model and the data model and compare projections based on AUS, POD, and DMD. Using these projected models, we develop projected particle filter algorithms and apply them to the Lorenz'96 model (L96) and the Shallow Water Equations (SWE).

While the present paper is motivated in large part by techniques developed using AUS projections, see, e.g., [10], [12], [15], we are also motivated by recently developed techniques for localized particle filters. Recent work has often focused on the issue of localization, such as [16], and currently two localized particle filtering algorithms [17], [18] have been applied in an operational geophysical framework. In the localized particle filter of [18] observations are projected onto the subspace spanned by the ensemble of model forecasts to reduce the dimension of the observations. In [19], the authors apply a localized particle filter which reduces the number of particles needed for effective assimilation. In this scheme, particle weights are updated locally near observations, but are preserved away from observations to mimic the covariance localization in Ensemble Kalman Filter (EnKF). Other techniques such as the Dynamically Orthogonal formulation in [20], [21] can be interpreted in terms of reduced order physical and data models (see [22], [23], [24]).

Although the original contribution of this paper is in combining data-driven Reduced Order Models (ROM) with particle filters, both POD and DMD have been used in concert with other data assimilation techniques. Kalman filter assimilation with a DMD-ROM has been used to predict wind turbine wakes in [25], while in [26] the DMD was used to enhance a Bayesian-optimized Kalman filter to predict events in the upper atmosphere.

For medium- to high-dimensional models, POD can be used to determine the dominant energy modes and, by reducing the model to the corresponding subspace, exploit a possible low dimensional structure of the model space for use in the nonlinear filtering problem [27], with EnKF [28], and with the Four-dimensional variational data assimilation (4D-Var) assimilation scheme [29], [30], [31], [32]. Likewise, POD, tensorial POD, and discrete empirical interpolation have been used in the 4D-Var scheme to reduce the state space in application to the shallow water equations [33]. In [34], efficacy of assimilation based on merging DMD, neural networks, and 4D-Var was evaluated on chaotic dynamical systems.

While we primarily focus on using DMD and POD to enhance data assimilation algorithms, it is interesting to point out that [35], [36] used data assimilation techniques, specifically the Kalman filter and its variants, to enhance the computation of DMD, in order to reduce the contribution of system noise and lead to constructing better estimates on the eigenvalues corresponding to DMD modes.

In addition, our use of POD and DMD as dimension reduction techniques provides a bridge between techniques developed to assimilate coherent structures and the model reduction literature. Methods such as those developed in [37] and [38] assimilate coherent structures extracted from data, but without an explicit form for the likelihood of the coherent structures. Instead these works use likelihood-free sequential Monte Carlo methods, or an ad hoc perturbed observations approach. In this paper we derive an explicit likelihood that corresponds to coherent structures extracted by POD or DMD.

In section 2 we present background on data assimilation techniques including ensemble Kalman filters and particle filters. In section 3 we formulate, using abstract projections, the projected physical and data models and their use in the context of standard and optimal proposal particle filters. We outline in section 4 the basics of the POD, DMD, and AUS model reduction techniques. Section 5 contains numerical results obtained using the algorithms developed to take advantage of these projected physical and data models. These methods are applied to two nonlinear models: L96 and SWE. Numerical results show the efficacy of the techniques that have been developed. In section 6 we summarize and analyze our results and outline directions for future research.

Section snippets

Data assimilation: nonlinearity and non-Gaussian posteriors

Data assimilation is a suite of methods commonly used for obtaining accurate estimates of states and/or parameters associated with large-scale geophysical systems such as the climate and atmosphere. DA schemes seek to optimally combine the information contained in an observation yt, informed by collected data, with that in a forecast ut, given by a mathematical model of the system. The observation and forecast naturally contain associated error, due to factors such as instrumentation error in

Dimension reduction of state and observation

Consider the physical model (1) and the data model (2). There are two routes to dimension reduction using these models, namely physical model projection and data model projection. Recall that the physical state of the system is given by utRM and our observation data is given by ytRD. As previously discussed, the issue with geophysical models is that the dimensions of the physical and data space, M and D respectively, can be extremely large. This poses a problem with data assimilation methods,

Techniques for model reduction

Development of particle filters on a subspace of the state or observation, detailed in section 3, does not depend on any particular technique for computing the dimension reduction matrices Vt and Ut. In this section we outline three techniques for computing these matrices. POD and DMD are data-driven (model-free) techniques that only require a set of simulation snapshots to calculate the reduction subspace, while the Lyapunov Vectors (LV) computation requires access to derivatives of the

Numerical results

To evaluate the performance of Proj-OP-PF, we apply the presented techniques to two commonly used models: Lorenz'96 model (L96) and a configuration of the Shallow Water Equations (SWE) corresponding to a barotropic instability.

The experiments were chosen to evaluate how a particular choice of the order reduction technique, and the dimensions of reduced model and data dimensions, resp. Mq and Dq, influence the accuracy of assimilation, as well as protect the particle filter from weight collapse.

Discussion and conclusions

In this paper, we have derived a projected bootstrap and optimal proposal Particle Filters for physical and data models used in a standard data assimilation framework. Our focus is on state space based projections formed using Assimilation in the Unstable Subspace (AUS), Proper Orthogonal Decomposition (POD), or Dynamic Mode Decomposition (DMD). This framework provides a basis for employing Particle Filters for high dimensional nonlinear problems, and extensively tests a projected optimal

Acronyms

4D-Var
Four-dimensional variational data assimilation
AUS
Assimilation in the Unstable Subspace
DA
Data Assimilation
DMD
Dynamic Mode Decomposition
EnKF
Ensemble Kalman Filter
ESS
Effective Sample Size
KF
Kalman Filter
L96
Lorenz'96 model
LV
Lyapunov Vectors
ODE
ordinary differential equation
OP-PF
Optimal Proposal Particle Filter
PDE
partial differential equation
PDF
Probability Density Function
PF
Particle Filter
POD
Proper Orthogonal Decomposition
Proj-OP-PF
Projected Optimal Proposal Particle Filter
Proj-PF
Projected

References (78)

  • A. Chorin et al.

    Implicit particle filters for data assimilation

    Comput. Methods Appl. Mech. Eng.

    (2010)
  • P.J. van Leeuwen

    Nonlinear data assimilation in geosciences: an extremely efficient particle filter

    Q. J. R. Meteorol. Soc.

    (2010)
  • C. Snyder

    Particle filters, the “optimal” proposal and high-dimensional systems

  • C. Snyder et al.

    Obstacles to high-dimensional particle filtering

    Mon. Weather Rev.

    (2008)
  • P.J. van Leeuwen

    Aspects of particle filtering in high-dimensional spaces

  • A. Trevisan et al.

    Assimilation of standard and targeted observations within the unstable subspace of the observation–analysis–forecast cycle system

    J. Atmos. Sci.

    (2004)
  • F. Uboldi et al.

    Developing a dynamically based assimilation method for targeted and standard observations

    Nonlinear Process. Geophys.

    (2005)
  • A. Trevisan et al.

    On the Kalman filter error covariance collapse into the unstable subspace

    Nonlinear Process. Geophys.

    (2011)
  • L. Palatella et al.

    Lyapunov vectors and assimilation in the unstable subspace: theory and applications

    J. Phys. A, Math. Theor.

    (2013)
  • L. Palatella et al.

    Interaction of Lyapunov vectors in the formulation of the nonlinear extension of the Kalman filter

    Phys. Rev. E (3)

    (2015)
  • J. Maclean et al.

    Particle filters for data assimilation based on reduced order data models

    Q. J. R. Meteorol. Soc.

    (2021)
  • A. Carrassi et al.

    Controlling instabilities along a 3dvar analysis cycle by assimilating in the unstable subspace: a comparison with the enkf

    Nonlinear Process. Geophys.

    (2008)
  • M. Bocquet et al.

    Four-dimensional ensemble variational data assimilation and the unstable subspace

    Tellus, Ser. A Dyn. Meteorol. Oceanogr.

    (2017)
  • B. de Leeuw et al.

    Projected shadowing-based data assimilation

    SIAM J. Appl. Dyn. Syst.

    (2018)
  • A. Farchi et al.

    Comparison of local particle filters and new implementations

    Nonlinear Process. Geophys.

    (2018)
  • J. Poterjoy et al.

    Efficient assimilation of simulated observations in a high-dimensional geophysical system using a localized particle filter

    Mon. Weather Rev.

    (2016)
  • R. Potthast et al.

    A localized adaptive particle filter within an operational nwp framework

    Mon. Weather Rev.

    (2019)
  • J. Poterjoy

    A localized particle filter for high-dimensional nonlinear systems

    Mon. Weather Rev.

    (2016)
  • T. Sapsis

    Dynamically orthogonal field equations

    (2010)
  • T. Sondergaard et al.

    Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. Part I: Theory and scheme

    Mon. Weather Rev.

    (2013)
  • A.J. Majda et al.

    Blended particle filters for large-dimensional chaotic dynamical systems

    Proc. Natl. Acad. Sci. USA

    (2014)
  • G.V. Iungo et al.

    Data-driven Reduced Order Model for prediction of wind turbine wakes

    J. Phys. Conf. Ser.

    (2015)
  • P.M. Mehta et al.

    A new transformative framework for data assimilation and calibration of physical ionosphere-thermosphere models

    Space Weather

    (2018)
  • Z. Wang et al.

    Proper orthogonal decomposition method to nonlinear filtering problems in medium-high dimension

    IEEE Trans. Autom. Control

    (2020)
  • A.A. Popov et al.

    A multifidelity ensemble Kalman filter with reduced order control variates

    SIAM J. Sci. Comput.

    (2021)
  • Y. Cao et al.

    A reduced-order approach to four-dimensional variational data assimilation using proper orthogonal decomposition

    Int. J. Numer. Methods Fluids

    (2007)
  • G. Dimitriu et al.

    Comparative study with data assimilation experiments using proper orthogonal decomposition method

  • F. Fang et al.

    A POD reduced-order 4D-Var adaptive mesh ocean modelling approach

    Int. J. Numer. Methods Fluids

    (2009)
  • N. Panda et al.

    A data-driven non-linear assimilation framework with neural networks

    Comput. Geosci.

    (2021)
  • Cited by (0)

    This research was supported in part by NSF grants DMS-1714195 and DMS-1722578 and originated as part of the AIM 2019 MCRN summer school and academic year engagement program.

    View full text