Abstract

Large-scale education data are collected via complex sampling designs that incorporate clustering and unequal probability of selection. Multilevel models are often utilized to account for clustering effects. The probability weighted approach (PWA) has been frequently used to deal with the unequal probability of selection. In this study, we examine the performance of an intuitive, easy to implement approach named the sample distribution approach (SDA) that utilizes Markov Chain Monte Carlo (MCMC) methods and Bayesian inference. Our simulation design focused on clustering effects, represented by the Intraclass Correlation (ICC) and on the sample size of the cluster. We analyzed a large-scale educational assessment dataset (Early Childhood Longitudinal Study - Kindergarten 2011) to compute estimates for the simulation. Findings reveal that the SDA overall generated reliable posterior distributions of parameters and had small error variances. In addition, although design informativeness is important, the ICC and cluster sample size factors had little impact on the performance of this model-based approach.

摘要

大规模教育数据是通过包含聚类和不等选择概率的复杂抽样设计收集的。多级模型通常用于解释聚类效应。概率加权法（PWA）经常被用来处理不等概率的选择。在这项研究中，我们检验了一种名为样本分布方法 (SDA) 的直观、易于实现的方法的性能，该方法利用马尔可夫链蒙特卡罗 (MCMC) 方法和贝叶斯推理。我们的模拟设计侧重于以类内相关 (ICC) 为代表的聚类效应和聚类的样本大小。我们分析了一个大规模的教育评估数据集（幼儿纵向研究 - 幼儿园 2011 年）来计算模拟的估计值。研究结果表明，SDA 总体上产生了可靠的参数后验分布，并且误差方差很小。此外，尽管设计信息量很重要，但 ICC 和集群样本大小因素对这种基于模型的方法的性能影响不大。