Context:

Software testing is a pivotal activity in the development of high-quality software. As software is evolving through its life cycle, the need for a fault-revealing criterion assessing the effectiveness of the test suite grows. Over the years, researchers have proposed coverage-based criteria, including statement and branch coverage, to solve this issue. In literature, the effectiveness of such criteria is attested in terms of their correlations with the mutation score.

Objective:

In this paper, we aim at proposing a coverage-based criterion named statement frequency coverage, which outperforms statement and branch coverage in terms of correlation with mutation score.

Method:

To this end, we incorporated the frequency of executed statements into statement coverage and created a coverage-based criterion for assessing test suite effectiveness. Statement frequency coverage assigns a continuous value to a statement whose value is proportional to the number of times executed during test execution. We evaluated our approach on 22 real-world Python projects with more than 118 000 source lines of code (without blank lines, comments, and test cases) and 21 000 test cases through measuring the correlation between statement frequency coverage and corresponding mutation score.

Results:

The results show that statement frequency coverage outperforms statement and branch coverage criteria. The correlation between it and the corresponding mutation score is higher than the correlation of statement and branch coverage with their mutation score. The results also show that unlike statement and branch coverage, there is no statistical difference between statement frequency coverage and mutation score.

Conclusion:

Statement frequency coverage is a better choice compared to statement and branch coverage in assessing test suite effectiveness in the real-world setting. Furthermore, we demonstrate that although statement frequency coverage subsumes statement coverage, it is incomparable to branch coverage under the adequate test suite condition.

中文翻译：

---

声明频率覆盖率：用于评估测试套件有效性的代码覆盖率准则

内容：

软件测试是开发高质量软件的关键活动。随着软件生命周期的发展，对评估测试套件有效性的故障揭示标准的需求不断增长。多年来，研究人员提出了基于覆盖率的标准，包括声明和分支覆盖率，以解决此问题。在文献中，根据这些标准与突变评分的相关性证明了这些标准的有效性。

目的：

在本文中，我们旨在提出一种基于覆盖率的标准，称为语句频率覆盖率，该标准在与突变评分的相关性方面优于语句和分支覆盖率。

方法：

为此，我们将执行语句的频率合并到语句覆盖率中，并创建了一个基于覆盖率的标准来评估测试套件的有效性。语句频率覆盖范围为语句分配连续值，该值与测试执行期间执行的次数成正比。我们通过测量语句频率覆盖率和相应突变得分之间的相关性，在22个真实世界的Python项目中评估了我们的方法，这些项目具有超过11.8万个源代码行（无空行，注释和测试用例）和21000个测试用例。

结果：

结果表明，语句频率覆盖率优于语句覆盖率和分支覆盖率标准。它与相应的突变得分之间的相关性高于陈述和分支覆盖与其突变得分之间的相关性。结果还表明，与语句和分支覆盖率不同，语句频率覆盖率和突变得分之间没有统计差异。

结论：

与陈述和分支覆盖相比，陈述频率覆盖是评估真实环境中测试套件有效性的更好选择。此外，我们证明了尽管语句频率覆盖率包含语句覆盖率，但是在适当的测试套件条件下它与分支覆盖率是无与伦比的。