Skip to main content
SpringerLink
Log in

Bayesian optimization of functional output in inverse problems

  • Research Article
  • Published:
Optimization and Engineering Aims and scope Submit manuscript

Abstract

Motivated by the parameter identification problem of a reaction-diffusion transport model in a vapor phase infiltration processes, we propose a Bayesian optimization procedure for solving the inverse problem that aims to find an input setting that achieves a desired functional output. The proposed algorithm improves over the standard single-objective Bayesian optimization by (i) utilizing the generalized chi-square distribution as a more appropriate predictive distribution for the squared distance objective function in the inverse problems, and (ii) applying functional principal component analysis to reduce the dimensionality of the functional response data, which allows for efficient approximation of the predictive distribution and the subsequent computation of the expected improvement acquisition function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Alvarez MA, Rosasco L, Lawrence ND (2012) Kernels for vector-valued functions: a review. Found Trends® Mach Learn 4(3):195–266

    Article  Google Scholar 

  • Astudillo R, Frazier P (2019) Bayesian optimization of composite functions. In International Conference on Machine Learning, pages 354–363. PMLR

  • Azpitarte I, Knez M (2018) Vapor phase infiltration: from a bioinspired process to technologic application, a prospective review. MRS Commun 8(3):727–741. https://doi.org/10.1557/mrc.2018.126

    Article  Google Scholar 

  • Box GE, Hunter WG (1962) A useful method for model-building. Technometrics 4(3):301–318

    Article  Google Scholar 

  • Dancik GM, Dorman KS (2008) mlegp: statistical analysis for computer models of biological systems using R. Bioinformatics 24(17):1966–1967

    Article  Google Scholar 

  • Delaigle A, Hall P (2010) Defining probability density for a distribution of random functions. Annal Stat 38(2):1171–1193

    Article  MathSciNet  Google Scholar 

  • Duchesne P, de Micheaux PL (2010) Computing the distribution of quadratic forms: further comparisons between the liu-tang-zhang approximation and exact methods. Comput Stat Data Anal 54:858–862

    Article  MathSciNet  Google Scholar 

  • Frazier PI (2018) Bayesian optimization. INFORMS Tutorials, pages 255–278

  • Golub GH, Welsch JH (1969) Calculation of gauss quadrature rules. Math Comput 23(106):221–230

    Article  MathSciNet  Google Scholar 

  • Gutmann MU, Corander J (2016) Bayesian optimization for likelihood-free inference of simulator-based statistical models. J Mach Learn Res

  • Hines AL, Maddox RN (1985) Mass transfer: fundamentals and applications, vol 434. Prentice-Hall Englewood Cliffs, NJ

    Google Scholar 

  • Imhof J-P (1961) Computing the distribution of quadratic forms in normal variables. Biometrika 48(3/4):419–426

    Article  MathSciNet  Google Scholar 

  • Ingram WF, Jur JS (2019) Properties and applications of vapor infiltration into polymeric substrates. Jom 71(1):238–245. https://doi.org/10.1007/s11837-018-3157-9

    Article  Google Scholar 

  • Järvenpää M, Gutmann MU, Vehtari A, Marttinen P et al (2018) Gaussian process modelling in approximate bayesian computation to estimate horizontal gene transfer in bacteria. Annal Appl Stat 12(4):2228–2251

    MathSciNet  MATH  Google Scholar 

  • Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21(4):345–383

    Article  MathSciNet  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

    Article  MathSciNet  Google Scholar 

  • Joseph VR, Yan H (2015) Engineering-driven statistical adjustment and calibration. Technometrics 57(2):257–267

    Article  MathSciNet  Google Scholar 

  • Joseph VR, Gul E, Ba S (2015) Maximum projection designs for computer experiments. Biometrika 102(2):371–380

    Article  MathSciNet  Google Scholar 

  • Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Series B 63(3):425–464

    Article  MathSciNet  Google Scholar 

  • Kirsch A (2011) An introduction to the mathematical theory of inverse problems, vol 120. Springer Science & Business Media, Berlin

    Book  Google Scholar 

  • Kushner HJ (1964) A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J Basic Eng 86:97–106

  • Leng CZ, Losego MD (2017) Vapor phase infiltration (vpi) for transforming polymers into organic-inorganic hybrid materials: a critical review of current progress and future challenges. Mater Horiz 4:747–771. https://doi.org/10.1039/C7MH00196G

    Article  Google Scholar 

  • Liu H, Tang Y, Zhang HH (2009) A new chi-square approximation to the distribution of non-negative definite quadratic forms in non-central normal variables. Comput Stat Data Anal 53(4):853–856

    Article  MathSciNet  Google Scholar 

  • Loeppky JL, Sacks J, Welch WJ (2009) Choosing the sample size of a computer experiment: A practical guide. Technometrics 51(4):366–376

    Article  MathSciNet  Google Scholar 

  • Losego MD, Peng Q (2019) Atomic Layer Deposition and Vapor Phase Infiltration, chapter 5, pages 135–159. John Wiley & Sons, Ltd. ISBN 9783527819249. https://doi.org/10.1002/9783527819249.ch5

  • Mak S, Joseph VR (2018) Support points. Annal Stat 46(6A):2562–2592

    Article  MathSciNet  Google Scholar 

  • Marttinen P, Croucher NJ, Gutmann MU, Corander J, Hanage WP (2015) Recombination produces coherent bacterial species clusters in both core and accessory genomes. Microb Genomics. https://doi.org/10.1099/mgen.0.000038

    Article  Google Scholar 

  • Matsui K, Kusakawa S, Ando K, Kutsukake K, Ujihara T, Takeuchi I (2019) Bayesian active learning for structured output design. arXiv preprint arXiv:1911.03671

  • Mockus J (1994) Application of bayesian approach to numerical methods of global and stochastic optimization. J Glob Optim 4(4):347–365

    Article  MathSciNet  Google Scholar 

  • Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313. https://doi.org/10.1093/comjnl/7.4.308 (01, ISSN 0010-4620.)

    Article  MathSciNet  MATH  Google Scholar 

  • Ramsay JO, Silverman BW (2004) Functional data analysis. Springer, New York

    MATH  Google Scholar 

  • Ramsay JO, Graves S, Hooker D (2020) fda: Functional Data Analysis. URL https://CRAN.R-project.org/package=fda. R package version 5.1.9

  • Ren Y, McGuinness EK, Huang C, Joseph VR, Lively RP, Losego MD (2021) Reaction-diffusion transport model to predict precursor uptake and spatial distribution in vapor-phase infiltration processes. Chem Mater 33(13):5210–5222. https://doi.org/10.1021/acs.chemmater.1c01283

    Article  Google Scholar 

  • Rusmassen C, Williams C (2006) Gaussian process for machine learning. MIT Press, Cambridge

    Google Scholar 

  • Santner TJ, Williams BJ, Notz WI (2018) The design and analysis of computer experiments. Springer, New York

    Book  Google Scholar 

  • Subramanian A, Tiwale N, Nam C-Y (2019) Review of recent advances in applications of vapor-phase material infiltration based on atomic layer deposition. JOM 71(1):185–196. https://doi.org/10.1007/s11837-018-3141-4

    Article  Google Scholar 

  • Tan MH (2018) Gaussian process modeling of a functional output with information from boundary and initial conditions and analytical approximations. Technometrics 60(2):209–221

    Article  MathSciNet  Google Scholar 

  • Uhrenholt AK, Jensen BS (2019) Efficient bayesian optimization for target vector estimation. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2661–2670. PMLR

  • Waldman RZ, Mandia DJ, Yanguas-Gil A, Martinson ABF, Elam JW, Darling SB (2019) The chemical physics of sequential infiltration synthesis-a thermodynamic and kinetic perspective. J Chem Phys. https://doi.org/10.1063/1.5128108

    Article  Google Scholar 

  • Wood SN (2010) Statistical inference for noisy nonlinear ecological dynamic systems. Nature 466(7310):1102–1104

    Article  Google Scholar 

  • Xiong Y, Chen W, Apley D, Ding X (2007) A non-stationary covariance-based kriging method for metamodelling in engineering design. Int J Num Methods Eng 71(6):733–756

    Article  Google Scholar 

  • Xun X, Cao J, Mallick B, Maity A, Carroll RJ (2013) Parameter estimation of partial differential equation models. J Am Stat Assoc 108(503):1009–1020

    Article  MathSciNet  Google Scholar 

  • Zhao Y, Huo X, Mei Y (2021) Identification of underlying dynamic system from noisy data with splines. arXiv preprint arXiv:2103.10231

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation (DMREF-1921873). EKM was also supported by the Department of Defense (DoD) through the National Defense Science & Engineering Graduate Fellowship (NDSEG) Program.

Author information

Authors and Affiliations

  1. H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Chaofan Huang & V. Roshan Joseph

  2. School of Chemical and Biomolecular Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Yi Ren & Ryan P. Lively

  3. School of Materials Science and Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Emily K. McGuinness & Mark D. Losego

Authors
  1. Chaofan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emily K. McGuinness
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark D. Losego
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ryan P. Lively
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. V. Roshan Joseph
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Roshan Joseph.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 196 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, C., Ren, Y., McGuinness, E.K. et al. Bayesian optimization of functional output in inverse problems. Optim Eng 22, 2553–2574 (2021). https://doi.org/10.1007/s11081-021-09677-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11081-021-09677-1

Keywords

Mathematics Subject Classification

Search

Navigation