Objective

To present challenges in assessment of applied cognition, and the results of differential item functioning (DIF) analyses used to inform the development of a computerized adaptive test (CAT)

Design

Measurement evaluation cohort study. DIF analyses of 107 items were conducted across educational, age, and gender groups. DIF hypotheses informed the evaluation of the results.

Setting

Hospital-based rehabilitation from a single hospital system

Participants

2216 hospitalized patients

Main Outcome Measures

Applied cognition item pool from multiple sources

Results

Many items were hypothesized to show DIF, particularly for age. Information was moderately high in the lower (cognitive disability) tail of the distribution, but some items were not informative. Reliability estimates were high (> 0.89) across all studied groups, regardless of estimation method. There were 35 items with DIF of high magnitude, and 19 with accompanying supportive hypotheses.

Conclusions

A key clinical tool in inpatient rehabilitation medicine is assessment of applied functional cognitive ability to inform patient-centered rehabilitation strategies to improve function. This was the first study to evaluate measurement equivalence of the applied cognition item pool across large samples of hospitalized patients. Although about one third of the item pool evidenced DIF or low discrimination, results supported placement of most items into the bank and its use across groups differing in education, age, and gender. Six items were classified with salient DIF, defined as consistent DIF of high magnitude and or impact, with confirmatory directional DIF hypotheses, generated by content experts. These were recommended for adjustment or removal from the bank; four were deleted from the bank and two had lowered CAT exposure (administration frequency) rates. Many items hypothesized to show DIF contained content measuring constructs other than applied cognition such as physical frailty, perceptual difficulties, or skills reflective of greater educational attainment. Challenges in measurement of this construct are discussed.

中文翻译：

---

测量应用认知的挑战：急性护理 MCAT (FAMCAT) 应用认知项目库中功能评估的测量特性和等效性

客观的

提出应用认知评估的挑战，以及用于通知计算机化自适应测试 (CAT) 开发的差异项目功能 (DIF) 分析的结果

设计

测量评估队列研究。跨教育、年龄和性别群体对 107 个项目进行了 DIF 分析。DIF 假设为结果评估提供了信息。

环境

来自单一医院系统的以医院为基础的康复

参加者

2216 名住院患者

主要观察指标

来自多个来源的应用认知项目库

结果

许多项目被假设显示 DIF，特别是对于年龄。分布的较低（认知障碍）尾部的信息量适中，但有些项目没有提供信息。无论估计方法如何，所有研究组的可靠性估计都很高 (> 0.89)。有 35 个项目具有高强度的 DIF，19 个项目具有伴随的支持性假设。

结论

住院康复医学中的一个关键临床工具是评估应用功能认知能力，以告知以患者为中心的康复策略以改善功能。这是第一项评估住院患者大样本中应用认知项目库的测量等效性的研究。尽管大约三分之一的项目池证明了 DIF 或低歧视，但结果支持将大多数项目放入银行并在不同教育、年龄和性别的群体中使用。六个项目被归类为具有显着 DIF，定义为高度和/或影响的一致 DIF，具有由内容专家生成的确认性定向 DIF 假设。这些被建议调整或从银行中删除；四个从银行中删除，两个降低了 CAT 风险（管理频率）率。许多假设显示 DIF 的项目包含内容测量结构，而不是应用认知，例如身体虚弱、感知困难或反映更高教育程度的技能。讨论了测量此构造的挑战。