Bayesian optimization and Lipschitz optimization have developed alternative techniques for optimizing black-box functions. They each exploit a different form of prior about the function. In this work, we explore strategies to combine these techniques for better global optimization. In particular, we propose ways to use the Lipschitz continuity assumption within traditional BO algorithms, which we call Lipschitz Bayesian optimization (LBO). This approach does not increase the asymptotic runtime and in some cases drastically improves the performance (while in the worst case the performance is similar). Indeed, in a particular setting, we prove that using the Lipschitz information yields the same or a better bound on the regret compared to using Bayesian optimization on its own. Moreover, we propose a simple heuristics to estimate the Lipschitz constant, and prove that a growing estimate of the Lipschitz constant is in some sense “harmless”. Our experiments on 15 datasets with 4 acquisition functions show that in the worst case LBO performs similar to the underlying BO method while in some cases it performs substantially better. Thompson sampling in particular typically saw drastic improvements (as the Lipschitz information corrected for its well-known “over-exploration” pheonemon) and its LBO variant often outperformed other acquisition functions.

中文翻译：

---

结合贝叶斯优化和 Lipschitz 优化

贝叶斯优化和 Lipschitz 优化开发了用于优化黑盒函数的替代技术。他们每个人都利用了关于函数的不同形式的先验。在这项工作中，我们探索了将这些技术结合起来以实现更好的全局优化的策略。特别是，我们提出了在传统 BO 算法中使用 Lipschitz 连续性假设的方法，我们称之为 Lipschitz 贝叶斯优化 (LBO)。这种方法不会增加渐近运行时间，并且在某些情况下会大大提高性能（而在最坏的情况下，性能是相似的）。实际上，在特定设置中，我们证明，与单独使用贝叶斯优化相比，使用 Lipschitz 信息对遗憾产生相同或更好的限制。而且，我们提出了一种简单的启发式方法来估计 Lipschitz 常数，并证明对 Lipschitz 常数的不断增加的估计在某种意义上是“无害的”。我们在具有 4 个采集函数的 15 个数据集上的实验表明，在最坏的情况下，LBO 的性能与基础 BO 方法相似，而在某些情况下，它的性能要好得多。特别是 Thompson 采样通常会看到显着的改进（因为 Lipschitz 信息因其众所周知的“过度探索”现象而得到纠正）并且其 LBO 变体通常优于其他采集功能。