The accuracy of a prediction algorithm depends on contextual factors that may vary across deployment settings. To address this inherent limitation of prediction, we propose an approach to counterfactual prediction based on the g-formula to predict risk across populations that differ in their distribution of treatment strategies. We apply this to predict 5-year risk of mortality among persons receiving care for HIV in the U.S. Veterans Health Administration under different hypothetical treatment strategies. First, we implement a conventional approach to develop a prediction algorithm in the observed data and show how the algorithm may fail when transported to new populations with different treatment strategies. Second, we generate counterfactual data under different treatment strategies and use it to assess the robustness of the original algorithm’s performance to these differences and to develop counterfactual prediction algorithms. We discuss how estimating counterfactual risks under a particular treatment strategy is more challenging than conventional prediction as it requires the same data, methods, and unverifiable assumptions as causal inference. However, this may be required when the alternative assumption of constant treatment patterns across deployment settings is unlikely to hold and new data is not yet available to retrain the algorithm.

预测算法的准确性取决于可能因部署设置而异的上下文因素。为了解决预测的这种固有局限性，我们提出了一种基于 g 公式的反事实预测方法，以预测治疗策略分布不同的人群的风险。我们应用它来预测在不同的假设治疗策略下在美国退伍军人健康管理局接受 HIV 护理的人的 5 年死亡风险。首先，我们采用传统方法在观察到的数据中开发预测算法，并展示该算法在使用不同治疗策略转移到新人群时可能会如何失败。第二，我们在不同的处理策略下生成反事实数据，并用它来评估原始算法对这些差异的性能的稳健性，并开发反事实预测算法。我们讨论了在特定治疗策略下估计反事实风险如何比传统预测更具挑战性，因为它需要与因果推理相同的数据、方法和无法验证的假设。但是，当跨部署设置的恒定治疗模式的替代假设不太可能成立并且新数据尚不可用于重新训练算法时，可能需要这样做。我们讨论了在特定治疗策略下估计反事实风险如何比传统预测更具挑战性，因为它需要与因果推理相同的数据、方法和无法验证的假设。但是，当跨部署设置的恒定治疗模式的替代假设不太可能成立并且新数据尚不可用于重新训练算法时，可能需要这样做。我们讨论了在特定治疗策略下估计反事实风险如何比传统预测更具挑战性，因为它需要与因果推理相同的数据、方法和无法验证的假设。但是，当跨部署设置的恒定治疗模式的替代假设不太可能成立并且新数据尚不可用于重新训练算法时，可能需要这样做。