Abstract
We review Gaussian process regression (GPR) and analyze it in the context of Inverse Theory—the collection of techniques used in geophysics (among other fields) to understand the structure of data analysis problems and the quality of their solutions. By viewing GPR as a special case of generalized least squares (least squares with prior information), we derive expressions for a variety of standard Inverse Theory quantities, including the data and model resolution matrices, the importance (influence) vector, and the gradient of the solution with respect to a parameter. We study the impulse response in the one-dimensional continuum limit and provide formulas for its area and width. Finally, we demonstrate how the importance vector can be used to design an optimum GPR experiment, through a process we call importance winnowing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
This paper contains no data.
References
Abers G (1994) Three-dimensional inversion of regional P and S arrival times in the East Aleutians and sources of subduction zone gravity highs. J Geophys Res 99:4395–4412. https://doi.org/10.1029/93JB03107
Asante-Okyere S, Shen C, Yevenyo YY, Ziggah R, Zhu X (2018) Investigating the predictive performance of Gaussian process regression in evaluating reservoir porosity and permeability. Energies 11:3261. https://doi.org/10.3390/en11123261
Aster RC, Borchers B, Thurber CH (2018) Parameter estimation and inverse problems, 3rd edn. Elsevier, Waltham, p 404
Backus G, Gilbert F (1970) Uniqueness in the inversion of inaccurate gross earth data. Philos Trans R Soc Lond Ser A 266:123–192. https://doi.org/10.1098/rsta.1970.0005
Backus G, Gilbert F (1968) The resolving power of Gross Earth Data. Geophys J Roy Astron Soc 16:169–205. https://doi.org/10.1190/1.1444834
Bartels RH, Stewart GW (1972) Solution of the matrix equation AX + XB = C [F4]. Commun ACM 15:820–826. https://doi.org/10.1145/361573.361582
Belsey DA, Kuh E, Welch RE (2004) Data diagnostics: identifying influential data and souces or collinearity. Wiley, New Jersey
Bracewell RN (2000) The fourier transform and its applications, 3rd edn. McGraw-Hill, Boston, p 616
Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted ℓ1 minimization. J Fourier Anal Appl 14:877–905. https://doi.org/10.1007/s00041-008-9045-x
Chen P, Jordan TH, Lee E-J (2010) Perturbation kernels for generalized seismological data functionals (GSDF). Geophys J Int 183:869–883. https://doi.org/10.1111/j.1365-246X.2010.04758.x]
Gentile R, Galasso C (2020) Gaussian process regression for seismic fragility assessment of building portfolios. Struct Saf 87:101980. https://doi.org/10.1016/j.strusafe.2020.101980
Grubbs FGE (1969) Procedures for detecting outlying observations in samples. Technometrics 11(1):1–21. https://doi.org/10.1080/00401706.1969.10490657
De Gruttola V, Ware J, Louis T (1987) Influence analysis of generalized least squares estimators. J Am Stat Assoc 82:911–917. https://doi.org/10.2307/2288804
Hanse TM, Journel AG, Tarantola A, Mosegaard K (2006) Linear inverse Gaussian theory and geostatistics. Geophysics. https://doi.org/10.1190/1.2345195
Hansen PC (1992) Analysis of discrete Ill-posed problems by means of the L-curve. SIAM Rev 34:561–580. https://doi.org/10.1137/1034115
Hines TT, Hetland EA (2018) Revealing transient strain in geodetic data with Gaussian process regression. Geophys J Int 212:2116–2130. https://doi.org/10.1093/gji/ggx525
Kopp RE, Horton RM, Little CM, Mitrovica JX, Oppenheimer M, Rasmussen DJ, Strauss BH, Tebaldi C (2014) Probabilistic 21st and 22nd century sea-level projections at a global network of tide-gauge sites. Earth’s future 2(8):383–406. https://doi.org/10.1002/2014EF000239
Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J Chem, Metall Min Soc 52:119–139
Lawson C, Hanson R (1974) Solving least squares problems. Prentice-Hall, Prentice, p 337
Levenberg K (1944) A method for the solution of certain non-linear problems in least-squares. Q Appl Math 2:164–168
Li RC (2014) Matrix perturbation theory, in Hogben, Handbook of linear algebra, 2nd edn. CRC Press, London, p 1402
Malinverno A, Briggs VA (2004) Expanded uncertainty quantification in inverse problems: Hierarchical Bayes and empirical Bayes. Geophysics 69:877–1103. https://doi.org/10.1190/1.1778243
Menke W (1984) Geophysical data analysis: discrete inverse theory. Academic Press, New York
Menke W (1991) Application of the POCS inversion method to interpolating topography and other geophysical fields. Geophys Res Lett 18:435–438. https://doi.org/10.1029/90GL00343
Menke W (2014) Review of the generalized least squares method. Surv Geophys 36:1–25. https://doi.org/10.1007/s10712-014-9303-1
Menke W (2018) Geophysical data analysis: discrete inverse theory, 4th edn. Academic Press, Elsevier, p 350
Menke W, Eilon Z (2015) Relationship between data smoothing and the regularization of inverse problems. Pure Appl Geophys 172:2711–2726. https://doi.org/10.1007/s00024-015-1059-0
Menke W, Menke J (2016) Environmental Data Analysis with MATLAB, 2nd edn. Academic Press, Elsevier, p 342
Menke W (2021) Tuning of Prior Covariance in Generalized Least Squares. Appl Math 12:157–170. https://doi.org/10.4236/am.2021.123011
Neal, RM (1994) Priors of infinite networks. Technical Report CRG-TR-94-1, Department of Computer Science, University of Toronto (Toronto, Canada).
Nooria M, Hassani H, Javaheriana A, Amindavar H, Torabi S (2019) Automatic fault detection in seismic data using Gaussian process regression. J Appl Geophys 163:117–131. https://doi.org/10.1016/j.jappgeo.2019.02.018
Parker RL (1994) Geophysical inverse theory. Princeton University Press, Princeton, p 386
Petersen KB, Pedersen MS (2008) The Matrix Cookbook, 71p, https://thematrixcookbook.com.
Piecuch CG, Huybers P, Tingley MP (2017) Comparison of full and empirical bayes approaches for inferring sea-level changes from tide-gauge data. J Geophys Res 122:2243–2258. https://doi.org/10.1002/2016JC012506
Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes 3rd edition: the art of scientific computing. Cambridge University Press, New York, p 1235
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, Cambridge, p 272
Ray A, Myer D (2019) Bayesian geophysical inversion with trans-dimensional Gaussian process machine learning. Geophys J Int 217:1706–1726. https://doi.org/10.1093/gji/ggz111
Reid A., O'Callaghan ST, Bonilla EV, McCalman L, Rawling T, Ramos FT (2013) Bayesian joint inversions for the exploration of earth resources. In: Proceedings of the Twenty-Third International joint conference on artificial intelligence IJCAI '13, pp. 2877–2884, www.ijcai.org/Proceedings/2013.
Richardson RM, MacInnes SC (1989) The inversion of gravity data into three-dimensional polyhedral models. J Geophys Res 94:7555–7562. https://doi.org/10.1029/JB094iB06p07555
Shure L, Parker RL, Backus GE (1982) Harmonic splines for geomagnetic modelling. Phys Earth Planet Inter 28:215–222. https://doi.org/10.1016/0031-9201(82)90003-6
Smith WHF, Wessel P (1990) Gridding with continuous curvature splines in tension. Geophysics. https://doi.org/10.1190/11442837
Snyman JA, Wilke DN (2018) Practical mathematical optimization - basic optimization theory and gradient-based algorithms. springer optimization and its applications, 2nd edn. Springer, New York
Talagrand O, Courtier P (1987) Variational assimilation of meteorological observations with the adjoint vorticity equation. I: theory. Quart J Royal Meteorol Soc 113:1311–1328. https://doi.org/10.1002/qj.49711347812
Tarantola A, Valette B (1982a) Generalized non-linear inverse problems solved using the least squares criterion. Rev Geophys Space Phys 20:219–232
Tarantola A, Valette B (1982b) Inverse problems = quest for information. J Geophys 50:159–170
Tarantola A (2005) Inverse problem theory and methods for model parameter estimation, First Edition, SIAM: Society for Industrial and Applied Mathematics, p. 342, ISBN 13: 9780898715729.
Tikhonov AN (1943) On the stability of inverse problems. Dokl Akad Nauk SSSR 39:195–198
Trainor-Guitton WJ, Mukerji T, Knight R (2013) A methodology for quantifying the value of spatial information for dynamic Earth problems. Stoch Env Res Risk Assess 27:969–983. https://doi.org/10.1007/s00477-012-0619-4
Tromp J, Tape C, Liu Q (2005) Seismic tomography, adjoint methods, time reversal and banana-doughnut kernels. Geophys J Int 160:195–216. https://doi.org/10.1111/j.1365-246X.2004.02453.x
Webster R, Oliver MA (2007) Geostatistics for environmental scientists, 2nd edn. Hoboken, Wiley
Wiggins RA (1972) The general linear inverse problem: Implications of surface waves and free oscillations for Earth structure. Rev Geophys Space Phys 10:251–285. https://doi.org/10.1029/RG010i001p00251
Acknowledgements
This research was partially supported by the National Science Foundation under Grant EAR 20-02352.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflicts of interest with respect to this work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Menke, W., Creel, R. Gaussian Process Regression Reviewed in the Context of Inverse Theory. Surv Geophys 42, 473–503 (2021). https://doi.org/10.1007/s10712-021-09640-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10712-021-09640-w