Context:

Previous studies proposed different kinds of approaches for class integration test order generation, and corresponding systems can be implemented based on these approaches. Such class integration test order generation systems can facilitate the process of software integration testing if they are implemented correctly.

Objective:

However, a test oracle problem exists in the class integration test order generation systems. Since these approaches for class integration test order generation normally deliver a local optimum rather than a global optimum, there are no practically feasible ways to validate their generated class integration test orders, that is, these implementation systems are untestable.

Method:

To address the test oracle problem, we apply Metamorphic Testing (MT) to validate class integration test order generation systems. Metamorphic Relations (MRs), which are the key components of MT, reason about relations between test outputs of a system. Five effective MRs are developed to ensure the quality of the class integration test order generation systems. In these proposed MRs, follow-up test inputs are generated by modifying classes or class dependencies in the source test inputs while some characteristics of the source test outputs are preserved, for example, the same class integration test order or the equal stubbing cost. Faults can be detected in systems if an individual MR is violated for certain tests.

Results:

Failure detection of MT has been successfully demonstrated in empirical experiments on three systems implementing different typical class integration test order generation approaches. More than 84% of faulty programs can be detected by all MRs, for three class integration test order generation systems under investigation.

Conclusion:

The experimental results show that the proposed MRs are able to systematically and effectively detect faults in class integration test order generation systems. This study explores a new application domain in MT and further extends its applications in Software Engineering.

中文翻译：

---

使用“变形测试”验证类集成测试订单生成系统

内容：

先前的研究提出了用于类别集成测试订单生成的不同方法，并且可以基于这些方法来实现相应的系统。如果正确实现，则此类类集成测试订单生成系统可以促进软件集成测试的过程。

目的：

但是，在类集成测试订单生成系统中存在一个测试预言问题。由于这些用于类集成测试订单生成的方法通常会提供局部最优而不是全局最优，因此没有实际可行的方法来验证其生成的类集成测试订单，也就是说，这些实现系统是不可测试的。

方法：

为了解决测试oracle问题，我们应用了Metamorphic Testing（MT）来验证类集成测试订单生成系统。作为MT的关键组成部分的变质关系（MR）是系统测试输出之间关系的原因。开发了五个有效的MR，以确保类集成测试订单生成系统的质量。在这些提议的MR中，通过修改源测试输入中的类或类相关性来生成后续测试输入，同时保留源测试输出的某些特征，例如，相同的类集成测试顺序或相等的存根成本。如果某些测试违反了单个MR，则可以在系统中检测到故障。

结果：

MT的故障检测已在实施不同典型类集成测试订单生成方法的三个系统的经验实验中成功证明。对于正在研究的三类集成测试订单生成系统，所有MR都可以检测到超过84％的故障程序。

结论：

实验结果表明，提出的MR能够系统有效地检测类集成测试订单生成系统中的故障。这项研究探索了MT中的新应用领域，并进一步扩展了其在软件工程中的应用。