Most machine learning methods require careful selection of hyper-parameters in order to train a high performing model with good generalization abilities. Hence, several automatic selection algorithms have been introduced to overcome tedious manual (try and error) tuning of these parameters. Due to its very high sample efficiency, Bayesian Optimization over a Gaussian Processes modeling of the parameter space has become the method of choice. Unfortunately, this approach suffers from a cubic compute complexity due to underlying Cholesky factorization, which makes it very hard to be scaled beyond a small number of sampling steps. In this paper, we present a novel, highly accurate approximation of the underlying Gaussian Process. Reducing its computational complexity from cubic to quadratic allows an efficient strong scaling of Bayesian Optimization while outperforming the previous approach regarding optimization accuracy. The first experiments show speedups of a factor of 162 in single node and further speed up by a factor of 5 in a parallel environment.

中文翻译：

---

使用惰性高斯过程的可扩展超参数优化

大多数机器学习方法需要仔细选择超参数，以训练具有良好泛化能力的高性能模型。因此，引入了几种自动选择算法来克服对这些参数进行繁琐的手动（尝试和错误）调整。由于其非常高的样本效率，贝叶斯优化对参数空间的高斯过程建模已成为首选方法。不幸的是，由于底层的 Cholesky 分解，这种方法受到三次计算复杂性的影响，这使得它很难扩展到少数采样步骤之外。在本文中，我们提出了一种新的、高度准确的基础高斯过程近似值。将其计算复杂度从三次减少到二次，可以有效地强缩放贝叶斯优化，同时在优化精度方面优于以前的方法。第一个实验表明，在单节点中加速了 162 倍，在并行环境中进一步加速了 5 倍。