BACKGROUND

Heart transplant programs and regulatory entities require highly accurate performance metrics to support internal quality improvement activities and national oversight of transplant programs, respectively. We assessed the accuracy of publicly reported performance measures.

METHODS

We used the United Network for Organ Sharing registry to study patients who underwent heart transplantation between January 1, 2016 and June 30, 2018. We used tests of calibration to compare the observed rate of 1-year graft failure to the expected risk of 1-year graft failure, which was calculated for each recipient using the July 2019 method published by the Scientific Registry of Transplant Recipients (SRTR). The primary study outcome was the joint test of calibration, which accounts for both the total number of events predicted (calibration-in-the-large) and dispersion of risk predictions (calibration slope).

RESULTS

6,528 heart transplants were analyzed. The primary test of calibration failed (p <0.0001), indicating poor accuracy of the SRTR model. The calibration-in-the-large statistic (0.63, 95% confidence interval [CI] 0.58-0.68, p < 0.0001) demonstrated overestimation of event rates while the calibration slope statistic (0.56, 95% CI 0.49-0.62, p <0.0001) indicated over-dispersion of event rates. Pre-specified subgroup analyses demonstrated poor calibration for all subgroups (each p <0.01). After recalibration, program-level observed/expected ratios increased by a median of 0.14 (p <0.0001).

CONCLUSIONS

Risk models employed for publicly-reported graft survival at U.S. heart transplant centers lack accuracy in general and in all subgroups tested. The use of disease-specific models may improve the accuracy of program performance metrics.

中文翻译：

---

用于心脏移植中心绩效公开报告的风险模型的准确性

背景

心脏移植计划和监管实体需要高度准确的绩效指标，以分别支持内部质量改进活动和移植计划的国家监督。我们评估了公开报告的绩效指标的准确性。

方法

我们使用器官共享联合网络研究了2016 年 1 月 1 日至 2018 年 6 月 30 日期间接受心脏移植的患者。我们使用校准测试来比较观察到的 1 年移植失败率与 1-年移植失败，这是使用移植受者科学登记处 (SRTR) 发布的 2019 年 7 月方法为每个受者计算的。主要研究结果是校准的联合测试，它说明了预测的事件总数（大校准）和风险预测的分散（校准斜率）。

结果

分析了 6,528 例心脏移植。校准的主要测试失败 ( p <0.0001)，表明 SRTR 模型的准确性较差。大规模校准统计量（0.63，95% 置信区间 [CI] 0.58-0.68，p < 0.0001）表明高估了事件发生率，而校准斜率统计量（0.56，95% CI 0.49-0.62，p <0.0001 ) 表示事件率的过度分散。预先指定的亚组分析表明所有亚组的校准较差（每个p <0.01）。重新校准后，计划级别的观察/预期比率增加了 0.14 的中位数（p <0.0001）。

结论

美国心脏移植中心用于公开报告的移植物存活率的风险模型总体上和所有测试的亚组都缺乏准确性。使用特定疾病模型可以提高项目绩效指标的准确性。