Evaluating the performance assessments of solvers (e.g., for computation programs), known as the solver benchmarking problem, has become a topic of intense study, and various approaches have been discussed in the literature. Such a variety of approaches exist because a benchmark problem is essentially a multicriteria problem. In particular, the appropriate multicriteria decision-making problem can correspond naturally to each benchmark problem and vice versa. In this study, to solve the solver benchmarking problem, we apply the ranking-theory method recently proposed for solving multicriteria decision-making problems. The benchmarking problem of differential evolution algorithms was considered for a case study to illustrate the ability of the proposed method. This problem was solved using ranking methods from different areas of origin. The comparisons revealed that the proposed method is competitive and can be successfully used to solve benchmarking problems and obtain relevant engineering decisions. This study can help practitioners and researchers use multicriteria decision-making approaches for benchmarking problems in different areas, particularly software benchmarking.

中文翻译：

---

多标准决策的排序方法：求解器和问题的基准测试的应用

评估求解器的性能评估（例如，对于计算程序），称为求解器基准测试问题，已成为一个深入研究的主题，并且在文献中讨论了各种方法。之所以存在如此多样的方法，是因为基准问题本质上是一个多标准问题。特别是，适当的多标准决策问题可以自然地对应每个基准问题，反之亦然。在这项研究中，为了解决求解器基准测试问题，我们应用了最近提出的排序理论方法来解决多标准决策问题。一个案例研究考虑了差分进化算法的基准问题，以说明所提出方法的能力。使用来自不同原产地的排名方法解决了这个问题。比较表明，所提出的方法具有竞争力，可以成功地用于解决基准问题并获得相关的工程决策。这项研究可以帮助从业者和研究人员使用多标准决策方法来对不同领域的问题进行基准测试，尤其是软件基准测试。