When scales or tests are used to make decisions about individuals (e.g., to identify which adults should be assessed for psychiatric disorders), it is crucial that these decisions be accurate and consistent. However, it is not obvious how to assess accuracy and consistency when the scale was administered only once to a given sample and the true condition based on the latent variable is unknown. This article describes a method based on the linear factor model for evaluating the accuracy and consistency of scale-based decisions using data from a single administration of the scale. We illustrate the procedure and provide R code that investigators can use to apply the method in their own data. Finally, in a simulation study, we evaluate how the method performs when applied to discrete (vs. continuous) items, a practice that is common in published literature. The results suggest that the method is generally robust when applied to discrete items.

当使用量表或测试对个人做出决定时（例如，确定哪些成年人应该接受精神疾病评估），这些决定的准确和一致至关重要。然而，当量表仅对给定样本进行一次且基于潜在变量的真实情况未知时，如何评估准确性和一致性并不明显。本文描述了一种基于线性因子模型的方法，用于使用来自量表的单次管理的数据来评估基于量表的决策的准确性和一致性。我们说明了该过程并提供了 R 代码，研究人员可以使用该代码在自己的数据中应用该方法。最后，在模拟研究中，我们评估该方法在应用于离散（与连续）项目时的表现，这是出版文献中常见的做法。结果表明，该方法在应用于离散项目时通常是稳健的。