Tests can be derived from extended finite state machine (EFSM) specifications considering the coverage of single-transfer faults, all transitions using a transition tour, all-uses, edge-pair, and prime path with side trip. We provide novel empirical assessments of the effectiveness of these test suites. The first assessment determines for each pair of test suites if there is a difference between the pair in covering EFSM faults of six EFSM specifications. If the difference is found significant, we determine which test suite outperforms the other. The second assessment is similar to the first; yet, it is carried out against code faults of 12 Java implementations of the specifications. Besides, two assessments are provided to determine whether test suites have better coverage of certain classes of EFSM (or code) faults than others. The evaluation uses proper data transformation of mutation scores and p-value adjustments for controlling Type I error due to multiple tests. Furthermore, we show that subsuming mutants have an impact on mutation scores of both EFSM and code faults; and accordingly, we use a score that removes them in order not to invalidate the obtained results. The assessments show that all-uses tests were outperformed by all other tests; transition tours outperformed both edge-pair and prime path with side trips; and single-transfer fault tests outperformed all other test suites. Similar results are obtained over the considered EFSM and code fault domains, and there were no significant differences between the test suites coverage of different classes of EFSM and code faults.

中文翻译：

---

针对基于模型和代码的故障评估扩展有限状态机的测试套件

测试可以从扩展的有限状态机 (EFSM) 规范中推导出来，考虑到单次转移故障的覆盖率、使用转移巡回的所有转移、所有使用、边对和带边行程的主要路径。我们对这些测试套件的有效性提供了新颖的经验评估。第一次评估确定每对测试套件在覆盖六个 EFSM 规范的 EFSM 故障方面是否存在差异。如果发现差异显着，我们确定哪个测试套件优于另一个。第二次评估与第一次类似；然而，它是针对规范的 12 个 Java 实现的代码错误执行的。此外，还提供了两个评估来确定测试套件是否比其他套件更好地覆盖了某些类别的 EFSM（或代码）故障。用于控制由于多次测试导致的 I 类错误的p值调整。此外，我们表明包含突变体对 EFSM 和代码错误的突变分数都有影响。因此，我们使用将它们删除的分数，以免使获得的结果无效。评估表明，所有其他测试都优于所有用途测试；过渡旅行的表现优于边对和主要路径的边路旅行；和单传输故障测试优于所有其他测试套件。在所考虑的 EFSM 和代码故障域上获得了类似的结果，并且不同类别的 EFSM 和代码故障的测试套件覆盖率之间没有显着差异。