Evaluation and benchmarking of many-objective optimization (MaOO) methods are complicated. The rapid development of new optimization algorithms for solving problems with many objectives has increased the necessity of developing performance indicators or metrics for evaluating the performance quality and comparing the competing optimization algorithms fairly. Further investigations are required to highlight the limitations of how criteria/metrics are determined and the consistency of the procedures with the evaluation and benchmarking processes of MaOO. A review is conducted in this study to map the research landscape of multi-criteria evaluation and benchmarking processes for MaOO into a coherent taxonomy. Then contentious and challenging issues related to evaluation are highlighted, and the performance of optimization algorithms for MaOO is benchmarked. The methodological aspects of the evaluation and selection of MaOO algorithms are presented as the recommended solution on the basis of four distinct and successive phases. First, in the determination phase, the evaluation criteria of MaOO are collected, classified and grouped for testing experts’ consensus on the most suitable criteria. Second, the identification phase involves the process of establishing a decision matrix via a crossover of the ‘evaluation criteria’ and MaOO’, and the level of importance of each selective criteria and sub-criteria from phase one is computed to identify its weight value by using the best–worst method (BWM). Third, the development phase involves the creation of a decision matrix for MaOO selection on the basis of the integrated BWM and VIKOR method. Last, the validation phase involves the validation of the proposed solution.

中文翻译：

---

多目标优化方法的多标准评估和基准测试过程的研究前景回顾：连贯分类法、挑战和推荐的解决方案

多目标优化 (MaOO) 方法的评估和基准测试很复杂。用于解决多目标问题的新优化算法的快速发展增加了开发用于评估性能质量和公平比较竞争优化算法的性能指标或度量的必要性。需要进一步调查以强调如何确定标准/指标的局限性，以及程序与 MaOO 的评估和基准过程的一致性。本研究进行了回顾，以将 MaOO 的多标准评估和基准测试过程的研究前景映射到一个连贯的分类法中。然后突出与评估相关的有争议和具有挑战性的问题，并对 MaOO 的优化算法的性能进行了基准测试。MaOO 算法的评估和选择的方法学方面是基于四个不同和连续的阶段的推荐解决方案。首先，在确定阶段，对MaOO的评估标准进行收集、分类和分组，以测试专家对最合适标准的共识。其次，识别阶段涉及通过“评估标准”和 MaOO 的交叉建立决策矩阵的过程，并计算第一阶段的每个选择标准和子标准的重要性级别，以通过以下方式识别其权重值使用最坏方法（BWM）。第三，开发阶段涉及基于集成 BWM 和 VIKOR 方法为 MaOO 选择创建决策矩阵。最后，验证阶段涉及对提议的解决方案的验证。