A recurring problem in software development is incorrect decision making on the techniques, methods and tools to be used. Mostly, these decisions are based on developers’ perceptions about them. A factor influencing people’s perceptions is past experience, but it is not the only one. In this research, we aim to discover how well the perceptions of the defect detection effectiveness of different techniques match their real effectiveness in the absence of prior experience. To do this, we conduct an empirical study plus a replication. During the original study, we conduct a controlled experiment with students applying two testing techniques and a code review technique. At the end of the experiment, they take a survey to find out which technique they perceive to be most effective. The results show that participants’ perceptions are wrong and that this mismatch is costly in terms of quality. In order to gain further insight into the results, we replicate the controlled experiment and extend the survey to include questions about participants’ opinions on the techniques and programs. The results of the replicated study confirm the findings of the original study and suggest that participants’ perceptions might be based not on their opinions about complexity or preferences for techniques but on how well they think that they have applied the techniques.

中文翻译：

---

对测试有效性的（错误）认知：实证研究

软件开发中经常出现的问题是对要使用的技术、方法和工具的错误决策。大多数情况下，这些决定是基于开发人员对它们的看法。影响人们感知的一个因素是过去的经验，但它不是唯一的。在这项研究中，我们旨在发现在缺乏先前经验的情况下，对不同技术的缺陷检测有效性的看法与其实际有效性的匹配程度。为此，我们进行了实证研究和复制。在最初的研究中，我们对学生进行了一项受控实验，他们应用了两种测试技术和一种代码审查技术。在实验结束时，他们进行了一项调查，以找出他们认为最有效的技术。结果表明，参与者的看法是错误的，这种不匹配在质量方面代价高昂。为了进一步了解结果，我们复制了受控实验并将调查扩展到包括有关参与者对技术和程序的意见的问题。重复研究的结果证实了原始研究的结果，并表明参与者的看法可能不是基于他们对复杂性的看法或对技术的偏好，而是基于他们认为自己应用这些技术的程度。