Fair comparisons of educational systems in large-scale assessments can be made only if the differences in curricula have little impact on the outcomes. This study investigated the sensitivity of science achievement rankings to varying degrees of curriculum implementation in the Trends in International Mathematics and Science Study (TIMSS). Country-specific teacher-reported curriculum implementation profiles across the TIMSS science domains were charted including their within-country variability across the classrooms for 33 participating countries of TIMSS 2015. A sensitivity test compared the original ranking to TIMSS curriculum implementation scenarios (a least-possible, a most-possible, and more realistic country-specific median implementation). In contrast to expectations, no support was found for a positive relationship between opportunity to learn and science achievement at the between-country level or the within-country level, with only minor exceptions. The sensitivity analysis under different curriculum implementation scenarios also suggests little impact on the rank order of the countries. Plausible explanations for this null finding are addressed; attention and research efforts should focus on improving the quality of curriculum implementation indicators in large-scale assessments.

中文翻译：

---

TIMSS国家科学成就排名对课堂学习机会差异的敏感性

只有在课程差异对成果影响不大的情况下，才能在大规模评估中公平地比较教育系统。这项研究调查了《国际数学和科学研究趋势》（TIMSS）中科学成就排名对不同程度课程实施的敏感性。绘制了TIMSS科学领域中针对特定国家的教师报告的课程实施情况图，包括TIMSS 2015的33个参与国在教室之间的国家内部差异。敏感性测试将原始排名与TIMSS课程实施情况进行了比较（最少，这是最可能且更切合实际的针对特定国家/地区的中位数实施方式）。与期望相反，在国家间或国家内部，没有发现学习机会与科学成就之间存在积极关系的支持，只有少数例外。不同课程实施方案下的敏感性分析也表明，对各国的排名影响不大。解决了这个无效发现的合理解释；注意和研究工作应集中在提高大规模评估中课程实施指标的质量上。