IntroductionThere is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation.MethodsWe studied pseudo‐values based on the Aalen‐Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR‐IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR‐IPCW). The MLR‐IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records.ResultsThe pseudo‐value, BLR‐IPCW, and MLR‐IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low‐density regions of predicted transition probability.ConclusionsWe recommend implementing either the pseudo‐value or BLR‐IPCW approaches to produce a calibration curve, combined with the MLR‐IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the “calibmsm” R package available on CRAN.

中文翻译：

---

多状态风险预测模型的校准图

简介目前没有关于如何评估用于风险预测的多状态模型的校准的指南。我们介绍了几种可用于为多状态模型的转移概率生成校准图的技术，然后通过模拟评估它们在存在随机和独立审查的情况下的性能。方法我们研究了基于 Aalen-Johansen 估计器的伪值，具有逆审查权重概率的二元逻辑回归（BLR-IPCW）和具有逆审查权重概率的多项逻辑回归（MLR-IPCW）。 MLR-IPCW 方法产生校准散点图，提供有关校准的额外见解。我们模拟了具有不同审查级别的数据，并评估了每种方法估计一组预测转移概率的校准曲线的能力。我们还开发了一个模型的校准评估，该模型预测来自关联的初级和二级医疗记录的一组患者的心血管疾病、2 型糖尿病和慢性肾病的发病率。结果伪值、BLR-IPCW 和 MLR-IPCW方法在随机审查下给出校准曲线的无偏估计。这些方法在存在独立审查的情况下基本上保持无偏差，即使审查机制与结果密切相关，偏差集中在预测转移概率的低密度区域。结论我们建议实施伪值或 BLR-IPCW 方法生成校准曲线，结合 MLR-IPCW 方法生成校准散点图。这些方法已被纳入 CRAN 上可用的“calibmsm”R 包中。