This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world. Promoting best practice in benchmarking is its main goal. The article discusses eight essential topics in benchmarking: clearly stated goals, well-specified problems, suitable algorithms, adequate performance measures, thoughtful analysis, effective and efficient designs, comprehensible presentations, and guaranteed reproducibility. The final goal is to provide well-accepted guidelines (rules) that might be useful for authors and reviewers. As benchmarking in optimization is an active and evolving field of research this manuscript is meant to co-evolve over time by means of periodic updates.

中文翻译：

---

优化中的基准测试：最佳实践和未解决的问题

这项调查汇集了来自世界各地不同背景的十多名研究人员的想法和建议。促进基准测试的最佳实践是其主要目标。本文讨论了基准测试中的八个基本主题：明确陈述的目标、明确指定的问题、合适的算法、适当的性能度量、深思熟虑的分析、有效和高效的设计、易于理解的演示以及保证的可重复性。最终目标是提供可能对作者和审稿人有用的广为接受的指南（规则）。由于优化中的基准测试是一个活跃且不断发展的研究领域，因此本手稿旨在通过定期更新随着时间的推移而共同发展。