Learning and correcting non-Gaussian model errors

Highlights

• A neural network (NN) approach for learning and correcting numerical model errors is proposed.

• NN modeling error correction is demonstrated for direct and inverse problems.

• Experimental and numerical corroboration supports the feasibility of NN model error compensation.

Abstract

All discretized numerical models contain modeling errors – this reality is amplified when reduced-order models are used. The ability to accurately approximate modeling errors informs statistics on model confidence and improves quantitative results from frameworks using numerical models in prediction, tomography, and signal processing. Further to this, the compensation of highly nonlinear and non-Gaussian modeling errors, arising in many ill-conditioned systems aiming to capture complex physics, is a historically difficult task. In this work, we address this challenge by proposing a neural network approach capable of accurately approximating and compensating for such modeling errors in augmented direct and inverse problems. The viability of the approach is demonstrated using simulated and experimental data arising from differing physical direct and inverse problems.

Introduction

Discretized numerical models, such as the spectral element and finite element method (FEM), are widely used to simulate physical systems that vary in space or space and time [1], [2]. Likely the most familiar use of discretized numerical methods is in solving partial differential equations (PDEs). The motivation for discretizing and solving PDEs numerically is broad but is largely centered on estimating solutions to problems with arbitrary geometry and boundary conditions that may otherwise be difficult to model analytically. Numerical solutions to PDEs also enrich understanding on the spatial-temporal evolution of complex physics, for example in applications of black hole dynamics [3], geophysical fluid dynamics [4], and elastoplasticity [5].

Of course, this flexibility comes at a cost – pervasive modeling errors. These errors, largely a consequence of simplifying assumptions and the discretization refinement itself [6], corrupt numerical solutions and are the source of frustration for many scientists and engineers. Adding to the complexity of this situation, users often need to weigh the fidelity and resolution of numerical solutions against the expectation of errors (which are, themselves, uncertain). To illustrate this phenomenon, consider a classical example where the FEM equipped with quadratic quadrilateral (Q8) elements is used to compute the displacement field of a fixed Timoshenko beam. To investigate the modeling errors, we can choose varying levels of discretization refinement (i.e. h refinement) and compare/benchmark to the analytical solution for the displacement field Δ as a function of the spatial coordinates – horizontal and vertical coordinates x and y, respectively. Using the analytical solution, the vertical displacements for a fictitious beam shown in Fig. 1(a) are given by $\mathrm{\Delta }(x,y=0)=\frac{F}{6EI}[x^2(3L-x)+\frac{x(5\nu +4)d^2}{4}]$ [7], where $F=2$ MN is the end load, $L=10$ m is the beam length, $E=200$ MPa is the modulus of elasticity, $\nu =0.33$ is the Poisson ratio, I is the moment of inertia, and $d=5$ m is the depth of the beam's rectangular cross section. We show the trial solutions to this problem in Fig. 1(b), where the accuracy of the finite element (FE) solutions are indicated by their deviations from the analytical solution and further quantified in 1(c) using a standard $L_2$ error norm plot. The illustrative problem here typifies the expected error convergence with increasing h refinement – i.e. as the mesh size densifies, the FE errors approach zero asymptotically. This realization reinforces the fact that, because FE sizes cannot be infinitesimal, numerical modeling errors are unavoidable (although, for this simple example, error convergence to machine precision is possible).

Not surprisingly, significant research has been conducted to reduce model errors dating back to the inception of computing in science and engineering, especially by those working in inverse problems where modeling errors result in artifacts in medical, geophysical, and material imaging [8]. Modeling errors are also a serious source of consideration for engineers using FE solutions to design infrastructure. For example, in some engineering cases, 5 or 10 percent modeling error may be tolerable; however, engineering judgment and discretion often play key roles in such a design process. Conversely, when the uncertainty of modeling error is sufficiently high or cannot be reliably estimated, minimizing errors may be the only option. Appropriately, researchers in these fields have developed a number of regimes for understanding modeling errors due to, e.g., the well-known physically-unrealistic zero energy modes in low-order elements and mesh-size dependence [9].

General approaches to approximating modeling errors – ultimately leading to their subtraction from the discretized solution – remain, however, very challenging [10]. Without question, authors such as Kaipio, Arridge, and coauthors have made substantial progress towards approximating assumed Gaussian modeling errors [11], [12], [13], while benefiting from improved-accuracy model reduction. Meanwhile, other modeling error approximation approaches have been effective in linear situations [14]. In cases where modeling errors are not linear as a function of the PDE solution or the error statistics are unknown a priori, available and accessible methods for approximating numerical modeling errors are sparse. First advances have been made for linear inverse problems, to incorporate an implicit model correction [15] within a learned gradient descent scheme [16]. Explicit corrections, as we investigate here, have been recently analyzed in [17], where the authors show that under sufficient approximation accuracy, we can expect to obtain solutions close to what we obtain with a correct model. We will take this as motivation and shall take the step to more demanding nonlinear inverse problems and verify the proposed techniques to experimental data. In particular, the approach discussed here provides an explicit quantification of the model error and not only a correction.

Considering first the realization that PDEs are themselves often highly nonlinear or that their solutions are nonlinearly dependent on the input parameters, one may quickly surmise that a “one size fits all” approach to quantifying modeling errors is possibly intractable. However, delving more deeply into how PDEs are numerically solved, one observes that there are two broad schools of approaches: namely, physics-based models and physics-informed models. The latter employs, e.g., neural networks (NNs) to develop surrogate models which emulate physics-based models and generate PDE solutions that are valid within the space of the physics-based training data. Such approaches have been the source of significant research in the past few years [18], [19], [20], [21]; this interest largely stems from the speed and accuracy of simulating physics with well-trained NNs. Accordingly, the success of NNs in developing nonlinear maps from causalities to PDE solutions leads to the ambition that NNs can also be used to develop effective nonlinear maps from PDE solutions to errors in PDE solutions. Assuming the former can be actualized, the use of NNs developed via deep learning for approximating numerical modeling errors has the distinct advantage that error statistics are not required a priori. It is worth remarking that, from a cautionary standpoint, the accuracy of NN predictions is heavily dependent on the training data. Accordingly, significant care should be taken when generating training data and assessing the reliability of NN predictions of modeling error.