The Cognitive Reflection Test (CRT) has become one of the most widely used psychological tests in decision research. It also acquired considerable fame in the news, social media, and classrooms around the globe. Given its popularity, it comes as no surprise that correct answers on the CRT questions can be easily found online. How does the availability of information about the test influence its validity as a measure of cognitive reflection, particularly, if it is deployed in an online study? We present evidence that participants searched for answers on the internet. We compared two CRT versions. The crucial difference was that correct responses could be easily found online for the standard CRT, but not its modified variant. In addition, we examined if performance-based payment affected test performance and participants’ propensity to cheat. The latter was operationalized as the online search for correct answers on the CRT questions. Our results suggested that cheating occurred during the experiment, and that doing so improved performance if correct responses were easily found on the web. When the CRT items were re-phrased so that they could not be found online, performance was not improved by incentives. Hence, incentives boosted cheating rather than cognitive reflection.

认知反射测试（CRT）已成为决策研究中使用最广泛的心理测试之一。它还在全球的新闻、社交媒体和课堂上享有盛誉。鉴于它的受欢迎程度，CRT 问题的正确答案很容易在网上找到也就不足为奇了。测试信息的可用性如何影响其作为认知反思衡量标准的有效性，特别是如果它被部署在在线研究中？我们提供了参与者在互联网上搜索答案的证据。我们比较了两个 CRT 版本。关键的区别在于可以很容易地在线找到标准 CRT 的正确响应，而不是其修改后的变体。此外，我们检查了基于绩效的支付是否会影响测试表现和参与者的作弊倾向。后者被操作为在线搜索 CRT 问题的正确答案。我们的结果表明，在实验过程中发生了作弊，如果在网络上很容易找到正确的答案，那么这样做可以提高性能。当 CRT 项目被重新措辞以至于无法在网上找到它们时，激励并没有提高性能。因此，激励促进了作弊而不是认知反思。