Identifying the root causes of test flakiness is one of the challenges faced by practitioners during software testing. In other words, the testing of the software is hampered by test flakiness. Since the research about test flakiness in large-scale software engineering is scarce, the need for an empirical case-study where we can build a common and grounded understanding of the problem as well as relevant remedies that can later be evaluated in a large-scale context is a necessity. This study reports the findings from a multiple-case study. The authors conducted an online survey to investigate and catalogue the root causes of test flakiness and mitigation strategies. We attempted to understand how practitioners perceive test flakiness in closed-source development, such as how they define test flakiness and what practitioners perceive can affect test flakiness. The perceptions of practitioners were compared with the available literature. We investigated whether practitioners' perceptions are reflected in the test artefacts such as what is the relationship between the perceived factors and properties of test artefacts. This study reported 19 factors that are perceived by professionals to affect test flakiness. These perceived factors are categorized as test code, system under test, CI/test infrastructure, and organization-related. The authors concluded that some of the perceived factors in test flakiness in closed-source development are directly related to non-determinism, whereas other perceived factors concern different aspects, for example, lack of good properties of a test case, deviations from the established processes, and ad hoc decisions. Given a data set from investigated cases, the authors concluded that two of the perceived factors (i.e., test case size and test case simplicity) have a strong effect on test flakiness.

中文翻译：

---

从业者对测试片状因素感知的实证分析

确定测试不稳定的根本原因是从业者在软件测试过程中面临的挑战之一。换句话说，软件的测试受到测试脆弱性的阻碍。由于对大规模软件工程中的测试脆弱性的研究很少，因此需要进行实证案例研究，我们可以建立对问题的共同和扎实的理解以及随后可以在大规模评估的相关补救措施上下文是必要的。本研究报告了多案例研究的结果。作者进行了一项在线调查，以调查和编目测试脆弱性和缓解策略的根本原因。我们试图了解从业者如何看待闭源开发中的测试脆弱性，例如他们如何定义测试片状性以及从业者认为会影响测试片状性的内容。从业者的看法与现有文献进行了比较。我们调查了从业者的感知是否反映在测试人工制品中，例如感知因素与测试人工制品的特性之间的关系是什么。这项研究报告了专业人士认为影响测试片状的 19 个因素。这些感知因素被归类为 这项研究报告了专业人士认为影响测试片状的 19 个因素。这些感知因素被归类为 这项研究报告了专业人士认为影响测试片状的 19 个因素。这些感知因素被归类为测试代码、被测系统、CI/测试基础设施以及与组织相关的. 作者得出结论，闭源开发中测试脆弱性的一些感知因素与非确定性直接相关，而其他感知因素涉及不同方面，例如，缺乏测试用例的良好特性，偏离既定流程和临时决定。给定来自调查案例的数据集，作者得出结论，两个感知因素（即测试用例大小和测试用例简单性）对测试脆弱性有很强的影响。