ABSTRACT

Parameter selection during the construction of surrogates is often conducted by minimizing the Mean Squared Cross-Validation Error (MSE-CV). Surrogates constructed using MSE are poorly optimized using gradient-based optimizers. Hence, Nelder–Mead like optimizers are often favoured, which is unfortunate as surrogates make analytical gradients freely available and gradient-based optimizers scale better with increasing dimension. To address this shortcoming, this article proposes a new Cross-Validation (CV) approach, by optimizing the surrogate and computing the Mean Optimizer Distance (MOD-CV) to the best design in the surrogate. Four experimental CV measures are compared on seven test problems and it is demonstrated that the performance of gradient-based optimizers can be significantly enhanced, with a possible 97% improvement in MOD-CV over MSE-CV using Sequential Least-Squares Quadratic Programming (SLSQP). Additionally, surrogates constructed using MOD-CV outperform surrogates constructed with MSE-CV, 80% of the time when optimized with SLSQP and 68% when optimized with Nelder-Mead.

摘要

代理构建过程中的参数选择通常通过最小化均方交叉验证误差 (MSE-CV) 来进行。使用 MSE 构建的代理使用基于梯度的优化器优化不佳。因此，Nelder-Mead 之类的优化器通常受到青睐，这是不幸的，因为代理使分析梯度免费可用，并且基于梯度的优化器随着维度的增加而更好地扩展。为了解决这个缺点，本文提出了一种新的交叉验证 (CV) 方法，通过优化代理并计算平均优化器距离 (MOD-CV) 到代理中的最佳设计。在七个测试问题上比较了四个实验性 CV 度量，结果表明可以显着提高基于梯度的优化器的性能，与使用顺序最小二乘二次规划 (SLSQP) 的 MSE-CV 相比，MOD-CV 可能有 97% 的改进。此外，使用 MOD-CV 构建的代理优于使用 MSE-CV 构建的代理，80% 的时间使用 SLSQP 优化，68% 的时间使用 Nelder-Mead 优化。