Abstract
Geostatistical models quantify spatial relations between model parameters and can be used to estimate and simulate properties away from known observations. The underlying statistical model, quantified through a joint probability density, most often consists of both an assumed statistical model and the specific choice of algorithm, including tuning parameters controlling the algorithm. Here, a theory is developed that allows one to compute the entropy of the underlying multivariate probability density when sampled using sequential simulation. The self-information of a single realization can be computed as the sum of the conditional self-information. The entropy is the average of the self-information obtained for many independent realizations. For discrete probability mass functions, a measure of the effective number of free model parameters, implied by a specific choice of probability mass function, is proposed. Through a few examples, the entropy measure is used to quantify the information content related to different choices of simulation algorithms and tuning parameters.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahmed NA, Gokhale D (1989) Entropy expressions and their estimators for multivariate distributions. IEEE Trans Inf Theory 35(3):688–692
Barfod AA, Møller I, Christiansen AV, Høyer AS, Hoffimann J, Straubhaar J, Caers J (2018) Hydrostratigraphic modeling using multiple-point statistics and airborne transient electromagnetic methods. Hydrol Earth Syst Sci 22(6):3351–3373
Bruen AA, Forcinito MA (2011) Cryptography, information theory, and error-correction: a handbook for the 21st century, vol 68. Wiley, New York
Cordua KS, Hansen TM, Mosegaard K (2015) Improving the pattern reproducibility of multiple-point-based prior models using frequency matching. Math Geosci 47(3):317–343
Cover TM, Thomas JA (2012) Elements of information theory. Wiley, New York
Daly C (2005) Higher order models using entropy, Markov random fields and sequential simulation. In: Leuangthong O, Deutsch CV (eds) Geostatistics Banff 2004. Springer, Dordrecht, pp 215–224
Deutsch CV, Journel AG (1998) GSLIB, geostatistical software library and user’s guide. Applied geostatistics, 2nd edn. Oxford University Press, Oxford
Devroye L (1986) Non-uniform random variate generation. Springer. 857 pp. ISBN 3-540-96305-7
Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Machine Intell 6:721–741
Goovaerts P (1997) Geostatistics for natural resources evalutaion, applied geostatistics series. Oxford University Press, Oxford
Goovaerts P (2009) Medical geography: a promising field of application for geostatistics. Math Geosci 41(3):243
Guardiano F, Srivastava R (1993) Multivariate geostatistics: beyond bivariate moments. Geostat Troia 1:133–144
Hansen TM, Mosegaard K (2008) VISIM: sequential simulation for linear inverse problems. Comput Geosci 34(1):53–76
Hansen TM, Mosegaard K, Cordua KS (2008) Using geostatistics to describe complex a priori information for inverse problems. In: 8th international geostatistics congress
Hansen TM, Cordua KS, Looms MC, Mosegaard K (2013) SIPPI: a Matlab toolbox for sampling the solution to inverse problems with complex prior information: part 1, methodology. Comput Geosci 52:470–480
Hansen TM, Cordua KS, Zunino A, Mosegaard K (2016a) Probabilistic integration of geo-information. In: Moorkamp M, Lelièvre PG, Linde N, Khan A (eds) Integrated imaging of the earth: theory and applications, vol 218. Wiley, New York, pp 93–116
Hansen TM, Vu LT, Bach T (2016b) MPSLIB: a c++ class for sequential simulation of multiple-point statistical models. SoftwareX 5:127–133
Høyer AS, Vignoli G, Hansen TM, Keefer DA, Jørgensen F et al (2017) Multiple-point statistical simulation for hydrogeological models: 3-d training image development and conditioning strategies. Hydrol Earth Syst Sci 21(12):6069
Journel AG (1996) Modelling uncertainty and spatial dependence: stochastic imaging. Int J Geogr Inf Syst 10(5):517–522
Journel A, Alabert F (1989) Non-Gaussian data expansion in the earth sciences. Terra Nova 1(2):123–134
Journel AG, Deutsch CV (1993) Entropy and spatial disorder. Math Geol 25(3):329–355
Journel AG, Huijbregts CJ (1978) Mining geostatistics, vol 600. Academic Press, London
Laloy E, Hérault R, Jacques D, Linde N (2018) Training-image based geostatistical inversion using a spatial generative adversarial neural network. Water Resour Res 54(1):381–406
Liu Y, Journel A (2004) Improving sequential simulation with a structured path guided by information content. Math Geol 36(8):945–964
Liu Y, Journel AG (2009) A package for geostatistical integration of coarse and fine scale data. Comput Geosci 35(3):527–547
MacKay DJ (2003) Information theory, inference and learning algorithms. Cambridge University Press, Cambridge
Mariethoz G, Caers J (2014) Multiple-point geostatistics algorithms. In: Mariethoz G, Caers J (eds) Multiple-point geostatistics: stochastic modeling with training images. Wiley, New York, pp 155–171
Metropolis N, Rosenbluth M, Rosenbluth A, Teller A, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21:1087–1092
Mustapha H, Dimitrakopoulos R (2011) Hosim: a high-order stochastic simulation algorithm for generating three-dimensional complex geological patterns. Compute Geosci 37(9):1242–1253
Oz B, Deutsch CV, Tran TT, Xie Y (2003) DSSIM-HR: a FORTRAN 90 program for direct sequential simulation with histogram reproduction. Comput Geosci 29(1):39–51 (ISSN 0098-3004)
Pyrcz MJ, Deutsch CV (2014) Geostatistical reservoir modeling. Oxford University Press, Oxford
Renard P, Allard D (2013) Connectivity metrics for subsurface flow and transport. Adv Water Resour 51:168–196
Reza FM (1994) An introduction to information theory. Courier Corporation, Chelmsford
Shannon CE (2001) (1948) A mathematical theory of communication. ACM SIGMOBILE Mob Comput Commun Rev. Reprint (2001) 5(1):3–55
Soares A (2001) Direct sequential simulation and cosimulation. Math Geol 33(8):911–926
Straubhaar J, Renard P, Mariethoz G, Froidevaux R, Besson O (2011) An improved parallel multiple-point algorithm using a list approach. Math Geosci 43(3):305–328
Strebelle S (2000) Sequential simulation drawing structures from training images. Ph.D. thesis, Stanford University
Strebelle S (2002) Conditional simulation of complex geological structures using multiple-point statistics. Math Geol 34(1):1–20
Tarantola A (2005) Inverse problem theory and methods for model parameter estimation. SIAM, Philadelphia (ISBN 0-89871-572-5)
Acknowledgements
This work was supported by a grant (#7017-00160B) from the Independent Research Fund Denmark. The author thanks Klaus Mosegaard and Knud Skou Cordua for many valuable discussions on the topic. All computations were performed using MPSlib (Hansen et al. 2016b) and the SIPPI MATLAB package (Hansen et al. 2013). The source code can be found at http://github.com/cultpenguin/sippi/entropy/.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hansen, T.M. Entropy and Information Content of Geostatistical Models. Math Geosci 53, 163–184 (2021). https://doi.org/10.1007/s11004-020-09876-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11004-020-09876-z