During testing, developers can place oracles externally or internally with respect to a method. Given a faulty execution state, i.e., one that differs from the expected one, an oracle might be unable to expose the fault if it is placed at a program point with no access to the incorrect program state or where the program state is no longer corrupted. In such a case, the oracle is subject to failed error propagation. We conducted an empirical study to measure failed error propagation on Defects4J, the reference benchmark for Java programs with real faults, considering all 6 projects available (386 real bugs and 459 fixed methods). Our results indicate that the prevalence of failed error propagation is negligible when testing is performed at the unit level. However, when system-level inputs are provided, the prevalence of failed error propagation increases substantially. This indicates that it is enough for method postconditions to predicate only on the externally observable state/data and that intermediate steps should be checked when testing at system level.

中文翻译：

---

具有实际错误的Java程序中错误传播失败的实证研究

在测试期间，开发人员可以将Oracle放在方法的外部或内部。给定一个错误的执行状态，即与预期的执行状态不同，如果oracle被放置在无法访问不正确的程序状态的程序点处，或者该程序状态不再被破坏，则可能无法暴露该错误。在这种情况下，预言将遭受错误传播的失败。考虑到所有可用的6个项目（386个实际错误和459个固定方法），我们进行了一项实证研究，以测量Defects4J（具有实际错误的Java程序的参考基准）上失败的错误传播。我们的结果表明，在单元级别进行测试时，错误传播失败的发生率可以忽略不计。但是，提供系统级输入后，错误传播失败的发生率大大增加。这表明方法后置条件仅基于外部可观察的状态/数据为前提就足够了，并且在系统级别进行测试时应检查中间步骤。