Abstract Automated test generation can reduce the manual effort in improving software quality. A test generation method employs code coverage, such as the widely-used branch coverage, to guide the inference of tests. These tests can be used to detect hidden faults. An automatic tool takes a specific type of code coverage as a configurable parameter. Given an automated tool of test generation, a fault may be detected by one type of code coverage, but omitted by another. In frequently released software projects, the time budget of testing is limited. Configuring code coverage for a testing tool can effectively improve the quality of projects. In this paper, we conduct a study on whether a fault can be detected by specific code coverage in automated test generation. We build predictive models with 60 metrics of faulty source code to identify detectable faults under eight types of code coverage. In the experiment, an off-the-shelf tool, EvoSuite is used to generate test data. Experimental results based on four research questions show that different types of code coverage result in the detection of different faults; a code coverage can be used as a supplement to increase the number of detected faults if another coverage is applied first; for each coverage, the number of detected faults increases with its cutoff time in test generation. Our result shows that the choice of code coverage can be learned via multi-objective optimization from sampled faults and directly applied to new faults. This study can be viewed as a preliminary result to support the configuration of code coverage in the application of automated test generation.

中文翻译：

---

能否检测到此故障：通过自动测试生成进行故障检测的研究

摘要 自动化测试生成可以减少提高软件质量的人工工作量。测试生成方法使用代码覆盖率，例如广泛使用的分支覆盖率，来指导测试的推理。这些测试可用于检测隐藏故障。自动工具将特定类型的代码覆盖率作为可配置参数。给定测试生成的自动化工具，故障可能会被一种代码覆盖率检测到，但被另一种代码覆盖率忽略。在频繁发布的软件项目中，测试的时间预算是有限的。为测试工具配置代码覆盖率可以有效提高项目质量。在本文中，我们对自动化测试生成中是否可以通过特定代码覆盖率检测到故障进行了研究。我们使用 60 个错误源代码指标构建预测模型，以识别八种代码覆盖率下的可检测错误。在实验中，使用现成的工具 EvoSuite 生成测试数据。基于四个研究问题的实验结果表明，不同类型的代码覆盖率导致检测到不同的故障；如果先应用另一个覆盖，代码覆盖可以作为补充以增加检测到的故障的数量；对于每个覆盖范围，检测到的故障数量随着其在测试生成中的截止时间而增加。我们的结果表明，代码覆盖率的选择可以通过多目标优化从采样故障中学习并直接应用于新故障。