This article shows how the concept of reliability of composite scores, as defined in classical test theory, can be extended to the context of multilevel modeling. In particular, it discusses the contributions and limitations of the various level-specific reliability indices proposed by Geldhof, Preacher, and Zyphur (2014), denoted as ω̃b and ω̃w (and also α̃b and α̃w). One major limitation of those indices is that they are quantities for latent, unobserved level-specific composite scores, and are not suitable for observed composites at different levels. As illustrated using simulated data in this article, ω̃b can drastically overestimate the true reliability of between-level composite scores (i.e., observed cluster means). Another limitation is that the development of those indices did not consider the recent conceptual development on construct meanings in multilevel modeling (Stapleton & Johnson, 2019; Stapleton, Yang, & Hancock, 2016). To address the second limitation, this article defines reliability indices (ω2l, ωb, ωw, α2l, αb, αw) for three types of multilevel observed composite scores measuring various multilevel constructs: individual, configural, shared, and within-cluster. The article also shows how researchers can obtain sample point and interval estimates using the derived formulas and the provided R and Mplus code. In addition, a large-scale national data set was used to illustrate the proposed methods for estimating reliability for the three types of multilevel composite scores, and practical recommendations on when different indices should be reported are provided. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

中文翻译：

---

多级数据的复合可靠性：关于观察分数和构造意义。

本文展示了如何将经典测试理论中定义的综合分数可靠性概念扩展到多级建模的上下文中。特别地，它讨论了 Geldhof、Preacher 和 Zyphur (2014) 提出的各种特定级别可靠性指标的贡献和局限性，表示为 ω̃b 和 ω̃w（以及 α̃b 和 α̃w）。这些指数的一个主要限制是它们是潜在的、未观察到的特定于水平的综合评分的数量，不适用于不同水平的观察到的综合。如本文使用模拟数据所示，ω̃b 可以大大高估级间综合得分（即观察到的聚类均值）的真实可靠性。另一个限制是，这些指数的发展没有考虑最近关于多层次建模中构造意义的概念发展（Stapleton & Johnson，2019 年；Stapleton、Yang 和 Hancock，2016 年）。为了解决第二个限制，本文为测量各种多层次结构的三种多层次观察综合得分定义了可靠性指数（ω2l、ωb、ωw、α2l、αb、αw）：个体、配置、共享和集群内。文章还展示了研究人员如何使用导出的公式和提供的 R 和 Mplus 代码获得样本点和区间估计。此外，还使用了一个大规模的国家数据集来说明所提出的三种多层次综合评分的可靠性估计方法，并提供了关于何时报告不同指数的实用建议。（PsycInfo 数据库记录 (c) 2020 APA，保留所有权利）。