Research on new optimization algorithms is often funded based on the motivation that such algorithms might improve the capabilities to deal with real-world and industrially relevant optimization challenges. Besides a huge variety of different evolutionary and metaheuristic optimization algorithms, also a large number of test problems and benchmark suites have been developed and used for comparative assessments of algorithms, in the context of global, continuous, and black-box optimization. For many of the commonly used synthetic benchmark problems or artificial fitness landscapes, there are however, no methods available, to relate the resulting algorithm performance assessments to technologically relevant real-world optimization problems, or vice versa. Also, from a theoretical perspective, many of the commonly used benchmark problems and approaches have little to no generalization value. Based on a mini-review of publications with critical comments, advice, and new approaches, this communication aims to give a constructive perspective on several open challenges and prospective research directions related to systematic and generalizable benchmarking for black-box optimization.

中文翻译：

---

元启发式黑盒优化的基准测试：前景和开放挑战

对新优化算法的研究通常是基于此类算法可能会提高处理现实世界和工业相关优化挑战的能力的动机而获得资助。除了大量不同的进化和元启发式优化算法外，还开发了大量测试问题和基准套件，用于在全局、连续和黑盒优化的背景下对算法进行比较评估。然而，对于许多常用的综合基准问题或人工适应度景观，没有可用的方法将结果算法性能评估与技术相关的现实世界优化问题联系起来，反之亦然。另外，从理论的角度来看，许多常用的基准问题和方法几乎没有泛化价值。基于对具有批判性评论、建议和新方法的出版物的小型评论，本次交流旨在对与黑盒优化的系统化和通用基准测试相关的几个开放挑战和前瞻性研究方向提供建设性的观点。