当前位置: X-MOL 学术Child Dev. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Methodological considerations for more robust and reliable developmental science: How historical conventions bias behavioral measurements
Child Development ( IF 5.661 ) Pub Date : 2023-12-08 , DOI: 10.1111/cdev.14051
Umay Sen 1 , Gustaf Gredebäck 1
Affiliation  

The mobile paradigm has been one of the most highly cited paradigms in developmental research (Rovee & Rovee, 1969; Rovee-Collier, 1997; Rovee-Collier et al., 1980). The impressive work of Carolyn Rovee-Collier, the founder of the paradigm, has altered the field's understanding of the early capacities of young infants, from reflex-based organisms to agents who actively construct their world (recently reviewed by Sen & Gredebäck, 2021). The paradigm suggested that young infants are exceptionally competent agents who learn selectively (Bhatt & Rovee-Collier, 1994; Gerhardstein et al., 1998), have a temporal understanding of various events (Gulya et al., 1998, 1999), form declarative memories (i.e., memories of events) (Rovee-Collier, 1997; Rovee-Collier & Hartshorn, 1999), and remember what they experienced for months in the first few months of life (Hartshorn et al., 1998; Rovee-Collier et al., 1999).

These key characteristics are what cultivated our fascination for a deeper understanding of the paradigm and its validity. Nevertheless, a thorough examination of the literature and methodological choices of the mobile paradigm unexpectedly pointed to some limitations (Sen & Gredebäck, 2021, 2022a). For example, null or contradictory findings are rare, while statistically significant results appear persistently. Furthermore, its methodology consisted of unique operational definitions of learning and memory, small sample sizes (e.g., six infants per group), and the use of a learning criteria that exclude infants that do not engage in the task as expected. To examine how these methodological choices affected prior results, we conducted a methodological integrity assessment with a focus on four common research biases. According to our findings, there was no systematic error regarding p-hacking, unintentional errors/fraud, or the use of opportunistic stopping rules. Based on this, we concluded that researchers who conducted these studies did so with scientific rigor. However, what we believe to be a serious issue regarding the reliability of the findings also emerged. The simulations that we ran to test the opportunistic use of researcher degrees of freedom demonstrated that the learning criteria and unique operationalizations of learning and memory had the potential to create false-positive rates significantly above chance level. The simulation results indicate a powerful, and frequent, bias in the mobile paradigm literature that jeopardizes its reliability. It highlights the importance of evaluating the claims regarding memory capacities in early infancy with caution.

In their commentary, the authors responded to our critique of their methodology by arguing that null and contradictory findings do exist in the mobile paradigm literature, that the use of learning criteria is both necessary and inconsequential, that the simulations lack ecological validity, and lastly that there is converging evidence regarding fundamental principles of memory development. In this article, we address these points individually and clarify why the methodological integrity assessment uncovers a reliability threat to the mobile paradigm literature focusing on memory development.



中文翻译:

更稳健、更可靠的发展科学的方法论考虑:历史惯例如何影响行为测量

移动范式一直是发展研究中被引用最多的范式之一(Rovee & Rovee,  1969 ; Rovee-Collier,  1997 ; Rovee-Collier et al.,  1980)。该范式的创始人 Carolyn Rovee-Collier 的令人印象深刻的工作改变了该领域对幼儿早期能力的理解,从基于反射的有机体到积极构建自己世界的代理人(Sen 和 Gredebäck 最近评论,  2021) 。该范式表明,年幼的婴儿是非常有能力的代理人,他们有选择性地学习(Bhatt&Rovee-Collier,  1994 Gerhardstein等人,  1998),对各种事件有时间理解(Gulya等人,  1998,1999 ,形成陈述性记忆(即事件的记忆)(Rovee-Collier,  1997;Rovee-Collier & Hartshorn,  1999),并记住他们在生命的最初几个月中经历的事情(Hartshorn 等,  1998;Rovee-Collier 等)等,  1999)。

这些关键特征激发了我们对更深入地理解该范式及其有效性的兴趣。然而,对移动范式的文献和方法选择的彻底检查意外地指出了一些局限性(Sen & Gredebäck,  2021,2022a 。例如,无效或矛盾的发现很少见,而统计上显着的结果却持续出现。此外,其方法包括学习和记忆的独特操作定义、小样本量(例如,每组六个婴儿)以及使用排除未按预期参与任务的婴儿的学习标准。为了研究这些方法学选择如何影响先前的结果,我们进行了方法学完整性评估,重点关注四种常见的研究偏差。根据我们的发现,不存在有关 p-hacking、无意错误/欺诈或使用机会主义停止规则的系统错误。基于此,我们得出结论,进行这些研究的研究人员是具有科学严谨性的。然而,我们认为调查结果可靠性的一个严重问题也出现了。我们为测试研究人员自由度的机会主义使用而进行的模拟表明,学习标准以及学习和记忆的独特操作有可能产生明显高于机会水平的假阳性率。模拟结果表明,移动范式文献中存在严重且频繁的偏见,这危及了其可靠性。它强调了谨慎评估有关婴儿早期记忆能力的说法的重要性。

在评论中,作者回应了我们对其方法论的批评,认为移动范式文献中确实存在无效且矛盾的发现,学习标准的使用既必要又无关紧要,模拟缺乏生态有效性,最后,关于记忆发展的基本原理,有一致的证据。在本文中,我们将分别解决这些问题,并阐明为什么方法完整性评估揭示了针对内存开发的移动范式文献的可靠性威胁。

更新日期:2023-12-08
down
wechat
bug