Abstract

The most common method of single-case data analysis is visual analysis, but interrater reliability among visual raters tends to be poor. A new paradigm of visual analysis is presented and tested with the goal of addressing this persistent limitation. In traditional visual analysis, graphs are viewed and rated one by one. However, in the ranking and pairwise comparison methods introduced here, graphs are compared to each other and sorted from least to most evidence of intervention effectiveness. Four visual raters scored a set of 30 previously published single-case graphs using a traditional rating method as well as the ranking and pairwise methods. As in previous studies of visual analysis, the raters failed to achieve acceptable interrater reliability with the traditional rating approach (α = 0.641). However, interrater reliability increased to satisfactory levels when graphs were scored with ranking (α = 0.847) and pairwise comparison (α = 0.860). Visual analysis scores based on the pairwise method were also used to evaluate the performance of three single-case effect size statistics.

摘要

单一案例数据分析的最常见方法是视觉分析，但视觉评估者之间的间信度往往很差。提出并测试了一种新的视觉分析范式，旨在解决这一持续存在的局限性。在传统的视觉分析中，图形是一幅被查看和定级的图形。但是，在此处介绍的排名和成对比较方法中，将图形进行相互比较，并从干预效果的最小到大多数证据进行排序。四个视觉评估者使用传统的评估方法以及排名和成对方法对一组30个先前发布的单例图进行了评分。与以前的视觉分析研究一样，评估者使用传统的评估方法（α= 0.641）未能获得可接受的跨度可靠性。然而，对图表进行排名（α= 0.847）和成对比较（α= 0.860）进行评分时，界面可靠性提高到令人满意的水平。基于成对方法的视觉分析得分也用于评估三个单例效应大小统计数据的性能。