In prediction model research, external validation is needed to examine an existing model's performance using data independent to that for model development. Current external validation studies often suffer from small sample sizes and consequently imprecise predictive performance estimates. To address this, we propose how to determine the minimum sample size needed for a new external validation study of a prediction model for a binary outcome. Our calculations aim to precisely estimate calibration (Observed/Expected and calibration slope), discrimination (C-statistic), and clinical utility (net benefit). For each measure, we propose closed-form and iterative solutions for calculating the minimum sample size required. These require specifying: (i) target SEs (confidence interval widths) for each estimate of interest, (ii) the anticipated outcome event proportion in the validation population, (iii) the prediction model's anticipated (mis)calibration and variance of linear predictor values in the validation population, and (iv) potential risk thresholds for clinical decision-making. The calculations can also be used to inform whether the sample size of an existing (already collected) dataset is adequate for external validation. We illustrate our proposal for external validation of a prediction model for mechanical heart valve failure with an expected outcome event proportion of 0.018. Calculations suggest at least 9835 participants (177 events) are required to precisely estimate the calibration and discrimination measures, with this number driven by the calibration slope criterion, which we anticipate will often be the case. Also, 6443 participants (116 events) are required to precisely estimate net benefit at a risk threshold of 8%. Software code is provided.

中文翻译：

---

用于对具有二元结果的临床预测模型进行外部验证的最小样本量


在预测模型研究中，需要使用独立于模型开发数据的外部验证来检查现有模型的性能。当前的外部验证研究通常样本量较小，因此预测性能估计不精确。为了解决这个问题，我们提出如何确定二元结果预测模型的新外部验证研究所需的最小样本量。我们的计算旨在精确估计校准（观察/预期和校准斜率）、歧视（C 统计）和临床效用（净效益）。对于每项测量，我们提出封闭式迭代解决方案来计算所需的最小样本量。这些需要指定：(i) 每个感兴趣估计的目标 SE（置信区间宽度），(ii) 验证群体中的预期结果事件比例，(iii) 预测模型的预期（错误）校准和线性预测变量值的方差验证人群中的情况，以及 (iv) 临床决策的潜在风险阈值。这些计算还可用于判断现有（已收集）数据集的样本大小是否足以进行外部验证。我们说明了我们对机械心脏瓣膜衰竭预测模型进行外部验证的建议，预期结果事件比例为 0.018。计算表明至少需要 9835 名参与者（177 个事件）来精确估计校准和歧视措施，该数字由校准斜率标准驱动，我们预计这种情况经常发生。此外，6443 名参与者（116 个事件）需要在 8% 的风险阈值下精确估计净收益。提供软件代码。