We examine the research protocols in Blair and Sambanis’ recent article on forecasting civil wars, where they argue that their theory-based model can predict civil war onsets better than several atheoretical alternatives or a model with country-structural factors. We find that there are several important mistakes and that their key finding is entirely conditional on the use of parametrically smoothed ROC curves when calculating accuracy, rather than the standard empirical ROC curves that dominate the literature. Fixing these mistakes results in a reversal of their claim that theory-based models of escalation are better at predicting onsets of civil war than other kinds of models. Their model is outperformed by several of the ad hoc, putatively non-theoretical models they devise and examine. More importantly, we argue that rather than trying to contrast the roles of theory and “atheoretical” machine learning in predictive modeling, it would be more productive to focus on ways in which predictive modeling and machine learning could be used to strengthen extant predictive work. Instead, we argue that predictive modeling and machine learning are effective tools for theory testing.

我们检查了 Blair 和 Sambanis 最近关于预测内战的文章中的研究协议，他们认为他们基于理论的模型可以比几种非理论替代方案或具有国家结构因素的模型更好地预测内战的爆发。我们发现有几个重要的错误，他们的关键发现完全取决于在计算精度时使用参数平滑的 ROC 曲线，而不是主导文献的标准经验 ROC 曲线。纠正这些错误导致他们声称基于理论的升级模型比其他类型的模型更能预测内战的爆发。他们的模型优于他们设计和研究的几个临时的、假定的非理论模型。更重要的是，我们认为，与其试图将理论和“非理论”机器学习在预测建模中的作用进行对比，不如专注于预测建模和机器学习可用于加强现有预测工作的方式。相反，我们认为预测建模和机器学习是理论测试的有效工具。