Bayesian Optimization (BO) is a surrogate-assisted global optimization technique that has been successfully applied in various fields, e.g., automated machine learning and design optimization. Built upon a so-called infill-criterion and Gaussian Process regression (GPR), the BO technique suffers from a substantial computational complexity and hampered convergence rate as the dimension of the search spaces increases. Scaling up BO for high-dimensional optimization problems remains a challenging task. In this paper, we propose to tackle the scalability of BO by hybridizing it with a Principal Component Analysis (PCA), resulting in a novel PCA-assisted BO (PCA-BO) algorithm. Specifically, the PCA procedure learns a linear transformation from all the evaluated points during the run and selects dimensions in the transformed space according to the variability of evaluated points. We then construct the GPR model, and the infill-criterion in the space spanned by the selected dimensions. We assess the performance of our PCA-BO in terms of the empirical convergence rate and CPU time on multi-modal problems from the COCO benchmark framework. The experimental results show that PCA-BO can effectively reduce the CPU time incurred on high-dimensional problems, and maintains the convergence rate on problems with an adequate global structure. PCA-BO therefore provides a satisfactory trade-off between the convergence rate and computational efficiency opening new ways to benefit from the strength of BO approaches in high dimensional numerical optimization.

中文翻译：

---

主成分分析辅助的高维贝叶斯优化

贝叶斯优化 (BO) 是一种代理辅助全局优化技术，已成功应用于各个领域，例如自动机器学习和设计优化。BO 技术建立在所谓的填充准则和高斯过程回归 (GPR) 之上，随着搜索空间维度的增加，其计算复杂性和收敛速度受到阻碍。为高维优化问题扩大 BO 仍然是一项具有挑战性的任务。在本文中，我们建议通过将 BO 与主成分分析 (PCA) 混合来解决 BO 的可扩展性，从而产生一种新颖的 PCA 辅助 BO (PCA-BO) 算法。具体来说，PCA 过程在运行期间从所有评估点学习线性变换，并根据评估点的可变性选择变换空间中的维度。然后，我们构建 GPR 模型，以及所选维度跨越的空间中的填充标准。我们根据 COCO 基准框架的多模态问题的经验收敛速度和 CPU 时间来评估 PCA-BO 的性能。实验结果表明，PCA-BO 可以有效减少高维问题的 CPU 时间，并在具有足够全局结构的问题上保持收敛速度。