ABSTRACT

Communication of International Large-Scale Assessment (ILSA) results is dominated by reporting average country achievement scores that conceal individual differences between pupils, schools, and items. Educational research primarily focuses on examining differences between pupils and schools, while differences between items are overlooked. Using a variance components model on the Programme for International Student Assessment (PISA) 2018 cognitive domains of reading, mathematics, and science literacy, we estimated how much of the response variation can be attributed to differences between pupils, schools, and items. The results show that uniformly across domains and countries, it mattered more for the correctness of an item response which items were responded to by a pupil (27–35%) than which pupil responded to these items (10–12%) or which school the pupil attended (5–7%). Given the findings, we argue that differences between items in ILSAs constitute a source of substantial untapped potential for secondary research.

摘要

国际大规模评估 (ILSA) 结果的交流主要是报告平均国家成绩分数，这些分数掩盖了学生、学校和项目之间的个体差异。教育研究主要关注学生和学校之间的差异，而忽略了项目之间的差异。使用国际学生评估计划 (PISA) 2018 年阅读、数学和科学素养认知领域的方差分量模型，我们估计了有多少响应变化可归因于学生、学校和项目之间的差异。结果表明，在各个领域和国家之间，与学生对这些项目的反应（10-12%）或学生就读的学校（5-7%）相比，学生对哪些项目的反应（27-35%）对项目反应的正确性更为重要。鉴于研究结果，我们认为 ILSA 项目之间的差异构成了二次研究的大量未开发潜力的来源。