Large-scale educational surveys, including PISA, often collect student ratings to assess teaching quality. Because of the sampling design in PISA, student ratings must be aggregated at the school level instead of the classroom level. To what extent does school-level aggregation of student ratings yield reliable and valid measures of teaching quality? We investigate this question for six scales measuring classroom management, emotional support, inquiry-based instruction, teacher-directed instruction, adaptive instruction, and feedback provided by PISA 2015. The sample consisted of 503,146 students from 17,678 schools in 69 countries/regions. Multilevel CFA and SEM were conducted for each scale in each country/region to evaluate school-level reliability (intraclass correlations 1 and 2), factorial validity, and predictive validity. In most countries/regions, school-level reliability was found to be adequate for the classroom management scale, but only low to moderate for the other scales. Examination of factorial and predictive validity indicated that the classroom management, emotional support, adaptive instruction, and teacher-directed instruction scales capture meaningful differences in teaching quality between schools. Meanwhile, the inquiry scale exhibited poor validity in almost all countries/regions. These findings suggest the possibility of using student ratings in PISA to investigate some aspects of school-level teaching quality in most countries/regions.

中文翻译：

---

学生评分是否提供了关于学校教学质量的可靠和有效信息？PISA 2015科学教学评估措施

包括 PISA 在内的大规模教育调查通常会收集学生评分以评估教学质量。由于 PISA 中的抽样设计，学生评分必须在学校层面而不是课堂层面进行汇总。学校级别的学生评分汇总在多大程度上产生了可靠和有效的教学质量衡量标准？我们从 6 个衡量课堂管理、情感支持、探究式教学、教师指导教学、适应性教学和 PISA 2015 提供的反馈的量表来调查这个问题。样本包括来自 69 个国家/地区的 17,678 所学校的 503,146 名学生。对每个国家/地区的每个量表进行多级 CFA 和 SEM 以评估学校级别的可靠性（班内相关性 1 和 2）、因子效度和预测效度。在大多数国家/地区，发现学校级别的可靠性对于课堂管理量表来说是足够的，但对于其他量表来说只有低到中等。对析因效度和预测效度的检验表明，课堂管理、情感支持、适应性教学和教师指导的教学量表捕捉到了学校之间教学质量的有意义的差异。同时，探究量表在几乎所有国家/地区都表现出较差的效度。这些调查结果表明，可以使用 PISA 中的学生评分来调查大多数国家/地区学校教学质量的某些方面。对析因效度和预测效度的检验表明，课堂管理、情感支持、适应性教学和教师指导的教学量表捕捉到了学校之间教学质量的有意义的差异。同时，探究量表在几乎所有国家/地区都表现出较差的效度。这些调查结果表明，可以使用 PISA 中的学生评分来调查大多数国家/地区学校教学质量的某些方面。对析因效度和预测效度的检验表明，课堂管理、情感支持、适应性教学和教师指导的教学量表捕捉到了学校之间教学质量的有意义的差异。同时，探究量表在几乎所有国家/地区都表现出较差的效度。这些调查结果表明，可以使用 PISA 中的学生评分来调查大多数国家/地区学校教学质量的某些方面。