The forced-choice method has been proposed as a viable strategy to prevent socially desirable responding (SDR) on self-report non-cognitive measures. The ability of the method to eliminate SDR stems from matching items that are perceived as equally desirable into forced-choice item-blocks. The gold standard in quantifying similarity between items in terms of desirability has been the “mean difference index”, that is, the absolute difference between items' mean desirability ratings. This index relies on the assumption that items have one true desirability value, as efficiently and unbiasedly estimated by their respective means, and may fail if this assumption does not hold. To circumvent this issue, we propose indexing similarity between items in terms of desirability with several robust measures of absolute agreement (i.e., inter-item agreement indices). Using an empirical example, we show that relying on the mean difference index may lead to suboptimal forced-choice item-block assembly by matching items with a relatively poor inter-item agreement with respect to desirability. R code for computing the proposed agreement indices on a set of desirability ratings is provided, as are recommendations for applied researchers.

强制选择方法已被提议作为一种可行的策略，以防止对自我报告非认知措施的社会期望响应 (SDR)。该方法消除 SDR 的能力源于将被认为同样需要的项目匹配到强制选择项目块中。在合意性方面量化项目之间相似性的黄金标准是“平均差异指数”，即项目平均合意性评级之间的绝对差异。该指数依赖于这样一个假设，即项目具有一个真实的合意性值，通过其各自的方法有效且无偏地估计，如果该假设不成立，则可能会失败。为了规避这个问题，我们建议使用几个可靠的绝对一致性度量（即，项目间一致性指数）。使用一个经验示例，我们表明依赖平均差异指数可能会导致次优的强制选择项目块组装，方法是将项目与在合意性方面相对较差的项目间一致性进行匹配。提供了用于计算一组合意性评级的建议一致性指数的 R 代码，以及对应用研究人员的建议。