For the test case prioritization problems, the average percent of faults detected (APFD) and its variant versions are widely used as metrics to evaluate prioritized test suite’s efficiency of fault detection. By a revisit of metrics for test case prioritization, we observe that APFD is only available for the scenarios where all test suites under evaluation contain the same number of test cases. Such a limitation is often overlooked, and lead to incorrect results when comparing fault detection efficiency of test suites with different sizes. Moreover, APFD cannot precisely illustrate the process of fault detection in the real world. Besides the APFD, most of its variants, including the NAPFD and the APFD[Formula: see text], have similar problems. This paper points out these limitations in detail by analyzing the physical explanation of APFD series metrics formally. In order to eliminate these limitations, we propose a series of improved metrics, including the relative average percent of faults detected (RAPFD) and the relative cost-cognizant weighted average percent of faults detected (RAPFD[Formula: see text]), to evaluate the efficiency of the test suite. Furthermore, for the scenario of parallel testing, a series of metrics including the relative average percent of faults detected in parallel testing ([Formula: see text]-RAPFD) and the relative cost-cognizant weighted average percent of faults detected in parallel testing ([Formula: see text]-RAPFD[Formula: see text]) are proposed too. All the proposed metrics refer to both the speed of fault detection and the constraint of the testing resource. A formal analysis and some examples show that all the proposed metrics provide much more precise illustrations of the fault detection process.

中文翻译：

---

重新审视测试用例优先级问题的指标

对于测试用例优先级问题，检测到的平均故障百分比 (APFD) 及其变体版本被广泛用作评估优先级测试套件的故障检测效率的指标。通过重新审视测试用例优先级的指标，我们观察到 APFD 仅适用于所有正在评估的测试套件包含相同数量的测试用例的场景。在比较不同大小的测试套件的故障检测效率时，这种限制经常被忽视，并导致结果不正确。此外，APFD 无法准确说明现实世界中的故障检测过程。除了 APFD，它的大部分变体，包括 NAPFD 和 APFD[公式：见正文]，都有类似的问题。本文通过形式化分析 APFD 系列度量的物理解释，详细指出了这些局限性。为了消除这些限制，我们提出了一系列改进的指标，包括检测到的故障的相对平均百分比（RAPFD）和检测到的故障的相对成本认知加权平均百分比（RAPFD[公式：见文本]），以评估测试套件的效率。此外，对于并行测试的场景，一系列指标包括并行测试中检测到的故障的相对平均百分比（[公式：见文本]-RAPFD）和并行测试中检测到的相对成本认知加权平均故障百分比（ [公式：见正文]-RAPFD[公式：见正文]）也被提出。所有提出的指标都涉及故障检测的速度和测试资源的约束。正式的分析和一些例子表明，所有提出的指标都为故障检测过程提供了更精确的说明。