Abstract
Bayesian MCMC calibration and uncertainty analysis for computationally expensive models is implemented using the SOARS (Statistical and Optimization Analysis using Response Surfaces) methodology. SOARS uses a radial basis function interpolator as a surrogate, also known as an emulator or meta-model, for the logarithm of the posterior density. To prevent wasteful evaluations of the expensive model, the emulator is built only on a high posterior density region (HPDR), which is located by a global optimization algorithm. The set of points in the HPDR where the expensive model is evaluated is determined sequentially by the GRIMA algorithm described in detail in another paper but outlined here. Enhancements of the GRIMA algorithm were introduced to improve efficiency. A case study uses an eight-parameter SWAT2005 (Soil and Water Assessment Tool) model where daily stream flows and phosphorus concentrations are modeled for the Town Brook watershed which is part of the New York City water supply. A Supplemental Material file available online contains additional technical details and additional analysis of the Town Brook application.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arnold, J. G., Srinivasan, R., Muttiah, R. R., and Williams, J. R. (1998), “Large Area Hydrologic Modeling and Assessment. Part I: Model Development,” Journal of the American Water Resources Association, 34, 73–89.
Bates, D. M., and Watts, D. G. (1988), Nonlinear Regression Analysis and its Applications, New York: Wiley.
Bayarri, M. J., Berger, J. O., Paulo, R., Sacks, J., Cafeo, J. A., Cavendish, C.-H., and Tu, J. (2007a), “A Framework for Validation of Computer Models,” Technometrics, 49, 138–154.
Bayarri, M. J., Berger, J. O., Cafeo, J., Garcia-Donato, G., Liu, F., Palomo, J., Parthasarathy, R. J., Paulo, R., Sacks, J., and Walsh, D. (2007b), “Computer Model Validation with Functional Output,” The Annals of Statistics, 35, 1874–1906.
Bliznyuk, N., Ruppert, D., and Shoemaker, C. A. (2011), “Bayesian Inference Using Efficient Interpolation of Computationally Expensive Densities With Variable Parameter Costs,” Journal of Computational and Graphical Statistics, 20, 636–655.
— (2012), “Local Derivative-Free Approximation of Computationally Expensive Posterior Densities,” Journal of Computational and Graphical Statistics. doi:10.1080/10618600.2012.681255.
Bliznyuk, N., Ruppert, D., Shoemaker, C. A., Regis, R., Wild, S., and Mugunthan, P. (2008), “Bayesian Calibration and Uncertainty Analysis for Computationally Expensive Models Using Optimization and Radial Basis Function Approximation,” Journal of Computational and Graphical Statistics, 17, 270–294.
Box, G. E. P., and Cox, D. R. (1964), “An Analysis of Transformations,” Journal of the Royal Statistical Society, Series B, 26, 211–246.
Buhmann, M. D. (2003), Radial Basis Functions, New York: Cambridge University Press.
Carroll, R. J., and Ruppert, D. (1984), “Power Transformation When Fitting Theoretical Models to Data,” Journal of the American Statistical Association, 79, 321–328.
— (1988), Transformation and Weighting in Regression, New York: Chapman & Hall.
Cumming, J. A., and Goldstein, M. (2009), “Small Sample Bayesian Designs for Complex High-Dimensional Models Based on Information Gained Using Fast Approximations,” Journal of the American Statistical Association, 51, 377–388.
Eckhardt, K., Haverkamp, S., Fohrer, N., and Frede, H. G. (2002), “SWAT-G, A Version of SWAT99.2 Modified for Application to Low Mountain Range Catchments,” Physics and Chemistry of the Earth, 27, 641–644.
Grizzetti, B., Bouraoui, F., Granlund, K., Rekolainen, S., and Bidoglio, G. (2003), “Modelling Diffuse Emission and Retention of Nutrients in the Vantaanjoki Watershed (Finland) Using the SWAT Model,” Ecological Modelling, 169, 25–38.
Hamilton, J. D. (1994), Time Series Analysis, Princeton: Princeton University Press.
Higdon, D., Kennedy, M., Cavendish, J. C., Cafeo, J. A., and Ryne, R. D. (2004), “Combining Field Data and Computer Simulations for Calibration and Prediction,” SIAM Journal of Scientific Computation, 26, 448–466.
Kennedy, M. C., and O’Hagan, A. (2001), “Bayesian Calibration of Computer Models,” Journal of the Royal Statistical Society, Series B, 63, 425–464.
Levy, S., and Steinberg, D. M. (2010), “Computer Experiments: A Review,” AStA Advances in Statistical Analysis, 94, 311–324.
Qian, P. Z. G. (2009), “Nested Latin Hypercube Design,” Biometrika, 96, 957–970.
Qian, P. Z. G., and Wu, C. F. J. (2008), “Bayesian Hierarchical Modeling for Integrating Low-Accuracy and High-Accuracy Experiments,” Technometrics, 50, 192–204.
Rasmussen, C. E. (2003), ````Gaussian Processes to Speed up Hybrid Monte Carlo for Expensive Bayesian Integrals,” in Bayesian Statistics 7, eds. J. M. Bernardo, J. O. Berger, A. P. Berger, and A. F. M. Smith, pp. 651–659.
Regis, R. G., and Shoemaker, C. A. (2009), “Parallel Stochastic Global Optimization Using Radial Basis Functions,” INFORMS Journal on Computing, 21, 411–426.
Ruppert, D., Wand, M. P., and Carroll, R. J. (2003), Semiparametric Regression, Cambridge: Cambridge University Press.
Santner, T. J., Williams, B. J., and Notz, W. I. (2010), The Design and Analysis of Computer Experiments, New York: Springer.
Shoemaker, C. A., Regis, R., and Fleming, R. (2007), “Watershed Calibration Using Multistart Local Optimization and Evolutionary Optimization With Radial Basis Function Approximation,” Journal of Hydrologic Science, 52, 450–465.
Singh, A. (2011), “Global Optimization of Computationally Expensive Hydrologic Simulation Models,” Ph.D. Thesis in Civil and Environmental Engineering, Cornell University.
Tierney, L. (1994), “Markov Chains for Exploring Posterior Distributions,” The Annals of Statistics, 22, 1701–1786.
Tierney, L., and Kadane, J. (1986), “Accurate Approximations for Posterior Moments and Marginal Densities,” Journal of the American Statistical Association, 81, 82–86.
Tjelmeland, H., and Hegstad, B. K. (2001), “Model Jumping Proposals in MCMC,” Scandinavian Journal of Statistics, 28, 205–223.
Tolson, B. (2005), “Automatic Calibration, Management and Uncertainty Analysis: Phosphorus Transport in the Cannonsville Watershed, ” Ph.D. dissertation, School of Civil and Environmental Engineering, Cornell University.
Tolson, B., and Shoemaker, C. A. (2004), “Watershed Modeling of the Cannonsville Basin Using SWAT2000: Model Development, Calibration and Validation for the Prediction of Flow, Sediment and Phosphorus Transport to the Cannonsville Reservoir, Version 1,” Technical Report, School of Civil and Environmental Engineering, Cornell University. Available at http://ecommons.library.cornell.edu/handle/1813/2710.
— (2007a), “The Dynamically Dimensioned Search Algorithm for Computationally Efficient Automatic Calibration of Environmental Simulation Models,” Water Resources Research, 43, W01413. doi:10.1029/2005WR004723.
— (2007b), “Cannonsville Reservoir Watershed SWAT2000 Model Development, Calibration and Validation,” Journal of Hydrology, 337, 68–89 doi:10.1016/j.jhydrol.2007.01.017.
Vanden Berghen, F., and Bersini, H. (2005), “CONDOR, a New Parallel, Constrained Extension of Powell’s UOBYQA Algorithm: Experimental Results and Comparison With the DFO Algorithm,” Journal of Computational and Applied Mathematics, 181, 157–175.
Wild, S. M., and Shoemaker, C. A. (2011), “Global Convergence of Radial Basis Functions Trust Region Derivative-Free Algorithms,” SIAM Journal on Optimization, 20, 387–415.
Wild, S. M., Regis, R. G., and Shoemaker, C. A. (2007), “ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions,” SIAM Journal on Scientific Computing, 30, 3197–3219.
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Ruppert, D., Shoemaker, C.A., Wang, Y. et al. Uncertainty Analysis for Computationally Expensive Models with Multiple Outputs. JABES 17, 623–640 (2012). https://doi.org/10.1007/s13253-012-0091-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13253-012-0091-0