Gaussian process emulators (GPE) are a machine learning approach that replicates computational demanding models using training runs of that model. Constructing such a surrogate is very challenging and, in the context of Bayesian inference, the training runs should be well invested. The current paper offers a fully Bayesian view on GPEs for Bayesian inference accompanied by Bayesian active learning (BAL). We introduce three BAL strategies that adaptively identify training sets for the GPE using information-theoretic arguments. The first strategy relies on Bayesian model evidence that indicates the GPE’s quality of matching the measurement data, the second strategy is based on relative entropy that indicates the relative information gain for the GPE, and the third is founded on information entropy that indicates the missing information in the GPE. We illustrate the performance of our three strategies using analytical- and carbon-dioxide benchmarks. The paper shows evidence of convergence against a reference solution and demonstrates quantification of post-calibration uncertainty by comparing the introduced three strategies. We conclude that Bayesian model evidence-based and relative entropy-based strategies outperform the entropy-based strategy because the latter can be misleading during the BAL. The relative entropy-based strategy demonstrates superior performance to the Bayesian model evidence-based strategy.

中文翻译：

---

使用信息论的高斯过程模拟器的 Bayesian3 主动学习


高斯过程模拟器 (GPE) 是一种机器学习方法，它使用模型的训练运行来复制计算要求较高的模型。构建这样的代理非常具有挑战性，并且在贝叶斯推理的背景下，应该充分投资训练运行。当前的论文提供了关于贝叶斯推理和贝叶斯主动学习（BAL）的 GPE 的完全贝叶斯观点。我们引入了三种 BAL 策略，它们使用信息论参数自适应地识别 GPE 的训练集。第一个策略依赖于贝叶斯模型证据，指示 GPE 与测量数据匹配的质量，第二个策略基于相对熵，指示 GPE 的相对信息增益，第三个策略基于信息熵，指示缺失信息在 GPE 中。我们使用分析基准和二氧化碳基准来说明我们的三种策略的性能。本文展示了与参考解决方案收敛的证据，并通过比较所介绍的三种策略来演示校准后不确定性的量化。我们得出的结论是，贝叶斯模型基于证据的策略和基于相对熵的策略优于基于熵的策略，因为后者在 BAL 过程中可能会产生误导。基于相对熵的策略表现出优于贝叶斯模型基于证据的策略的性能。