Complex models are often fitted to data using simulation-based calibration, a computationally challenging process. Several calibration methods to improve computational efficiency have been developed with no consensus on which methods perform best. We did a simulation study comparing the performance of 5 methods that differed in their Goodness-of-Fit (GOF) metrics and parameter search strategies. Posterior densities for two parameters of a simple Susceptible-Infectious-Recovered epidemic model were obtained for each calibration method under two scenarios. Scenario 1 (S1) allowed 60K model runs and provided two target statistics, whereas scenario 2 (S2) allowed 75K model runs and provided three target statistics. For both scenarios, we obtained reference posteriors against which we compare all other methods by running Rejection ABC for 5M parameter combinations, retaining the 0.1% best. We assessed performance by applying a 2D-grid to all posterior densities and quantifying the percentage overlap with the reference posterior. We considered basic and adaptive sampling calibration methods. Of the basic calibration methods, Bayesian calibration (Bc) Sampling Importance Resampling (S1:34.8%, S2: 39.8%) outperformed Rejection Approximate Bayesian Computation (ABC)(S1: 2.3%, S2: 1.8%). Among the adaptive sampling methods, Bc Incremental Mixture Importance Sampling (S1: 72.7%, S2: 85.5%) outperformed sequential Monte Carlo ABC (AbcSmc) (S1: 53.9%, S2: 72.9%) and Sequential ABC (S1: 21.6%, S2: 62.7%). Basic methods led to sub-optimal calibration results. Methods using the surrogate Likelihood as a GOF outperformed methods using a distance measure. Adaptive sampling methods were more efficient compared to their basic counterparts and resulted in accurate posterior distributions. BcIMIS was the best performing method. When three rather than two target statistics were available, the difference in performance between the adaptive sampling methods was less pronounced. Although BcIMIS outperforms the other methods, limitations related to the target statistics and available computing infrastructure may warrant the choice of an alternative method.

中文翻译：

---

根据数据校准模型：方法比较

复杂的模型通常使用基于模拟的校准来拟合数据，这是一个计算难题的过程。已经开发了几种提高计算效率的校准方法，但对于哪种方法效果最佳尚无共识。我们进行了一项仿真研究，比较了5种拟合度（GOF）指标和参数搜索策略不同的方法的性能。在两种情况下，针对每种校准方法，获得了一个简单的易感染恢复型流行病模型的两个参数的后验密度。方案1（S1）允许运行6万个模型并提供两个目标统计数据，而方案2（S2）允许75,000个模型运行并提供三个目标统计数据。对于这两种情况，我们获得了参考后验，我们通过对5M参数组合运行Rejection ABC与其他所有方法进行比较，保留了0.1％的最佳值。我们通过对所有后部密度应用2D网格并量化与参考后部的重叠百分比来评估性能。我们考虑了基本和自适应采样校准方法。在基本校准方法中，贝叶斯校准（Bc）抽样重要性重采样（S1：34.8％，S2：39.8％）优于拒绝近似贝叶斯计算（ABC）（S1：2.3％，S2：1.8％）。在自适应抽样方法中，Bc增量混合重要性抽样（S1：72.7％，S2：85.5％）优于顺序蒙特卡洛ABC（AbcSmc）（S1：53.9％，S2：72.9％）和顺序ABC（S1：21.6％， S2：62.7％）。基本方法导致次优的校准结果。使用替代可能性作为GOF的方法优于使用距离度量的方法。与基本采样方法相比，自适应采样方法效率更高，并且可以实现精确的后验分布。BcIMIS是效果最好的方法。当有三个而不是两个目标统计量可用时，自适应采样方法之间的性能差异就不太明显。尽管BcIMIS的性能优于其他方法，但与目标统计数据和可用的计算基础结构有关的限制可能会保证选择其他方法。当有三个而不是两个目标统计量可用时，自适应采样方法之间的性能差异就不太明显。尽管BcIMIS的性能优于其他方法，但与目标统计数据和可用的计算基础结构有关的限制可能会保证选择其他方法。当有三个而不是两个目标统计量可用时，自适应采样方法之间的性能差异就不太明显。尽管BcIMIS的性能优于其他方法，但与目标统计数据和可用的计算基础结构有关的限制可能会保证选择其他方法。