Abstract

It is common to include multiple dependent variables (DVs) in single-case experimental design (SCED) meta-analyses. However, statistical issues associated with multiple DVs in the multilevel modeling approach (i.e., possible dependency of error, heterogeneous treatment effects, and heterogeneous error structures) have not been fully investigated. In this study, we first addressed various issues caused by multiple DVs and examined current modeling practice, then proposed several modeling options for handling multiple DVs and compared their impact on parameter estimates and statistical inferences by conducting both empirical and simulation studies. The results indicated that different modeling options can lead to very different conclusions about the treatment effects, variance components, and model fit. Among the presented modeling options, modeling heterogeneity in the level-1 error structure and adding DV type as moderators had a noticeably large and consistent impact on both fixed and random effects as well as model fit. Although including DV types as an additional level had a relatively small impact compared to the other options, it was still able to alter the conclusion of the statistical inferences on the treatment effects.

摘要

在单一案例实验设计 (SCED) 荟萃分析中包含多个因变量 (DV) 是很常见的。然而，在多级建模方法中与多个 DV 相关的统计问题（即可能的误差依赖性、异质处理效果和异质错误结构）尚未得到充分研究。在这项研究中，我们首先解决了由多个 DV 引起的各种问题并检查了当前的建模实践，然后提出了几种处理多个 DV 的建模选项，并通过进行实证和模拟研究比较了它们对参数估计和统计推断的影响。结果表明，不同的建模选项可能导致关于治疗效果、方差分量和模型拟合的非常不同的结论。在提出的建模选项中，在 1 级误差结构中建模异质性并添加 DV 类型作为调节因子对固定效应和随机效应以及模型拟合都有明显的大而一致的影响。尽管与其他选项相比，将 DV 类型作为附加级别的影响相对较小，但它仍然能够改变对治疗效果的统计推断的结论。