Optimizing expensive black-box systems with limited data is an extremely challenging problem. As a resolution, we present a new surrogate optimization approach by addressing two gaps in prior research—unimportant input variables and inefficient treatment of uncertainty associated with the black-box output. We first design a new flexible non-interpolating parsimonious surrogate model using a partitioning-based multivariate adaptive regression splines approach, Tree Knot MARS (TK-MARS). The proposed model is specifically designed for optimization by capturing the structure of the function, bending at near-optimal locations, and is capable of screening unimportant input variables. Furthermore, we develop a novel replication approach called Smart-Replication, to overcome the uncertainty associated with the black-box output. The Smart-Replication approach identifies promising input points to replicate and avoids unnecessary evaluations of other data points. Smart-Replication is agnostic to the choice of a surrogate and can adapt itself to an unknown noise level. Finally to demonstrate the effectiveness of our proposed approaches we consider different complex global optimization test functions from the surrogate optimization literature. The results indicate that TK-MARS outperforms original MARS within a surrogate optimization algorithm and successfully detects important variables. The results also show that although non-interpolating surrogates can mitigate uncertainty, replication is still beneficial for optimizing highly complex black-box functions. The robustness and the quality of the final optimum solution found through Smart-Replication are competitive with that using no replications in environments with low levels of noise and using a fixed number of replications in highly noisy environments.

用有限的数据优化昂贵的黑盒系统是一个极具挑战性的问题。作为解决方案，我们通过解决先前研究中的两个空白——不重要的输入变量和对与黑盒输出相关的不确定性的低效处理，提出了一种新的替代优化方法。我们首先使用基于分区的多元自适应回归样条方法 Tree Knot MARS (TK-MARS) 设计了一个新的灵活的非插值简约代理模型。所提出的模型是专门为通过捕获函数的结构、在接近最佳位置弯曲而设计的优化，并且能够筛选不重要的输入变量。此外，我们开发了一种称为Smart-Replication的新型复制方法，以克服与黑盒输出相关的不确定性。智能复制方法识别出有希望复制的输入点，并避免对其他数据点进行不必要的评估。智能复制与代理的选择无关，可以适应未知的噪音水平。最后，为了证明我们提出的方法的有效性，我们考虑了替代优化文献中不同的复杂全局优化测试函数。结果表明，TK-MARS 在代理优化算法中优于原始 MARS，并成功检测到重要变量。结果还表明，尽管非插值代理可以减轻不确定性，但复制仍然有利于优化高度复杂的黑盒函数。