Electric sector capacity expansion models are widely used by academic, government, and industry researchers for policy analysis and planning. Many models overlap in their capabilities, spatial and temporal resolutions, and research purposes, but yield diverse results due to both parametric and structural differences. Previous work has attempted to identify some differences among commonly used capacity expansion models but has been unable to disentangle parametric from structural uncertainty. Here, we present a model benchmarking effort using highly simplified scenarios applied to four open-source models of the U.S. electric sector. We eliminate all parametric uncertainty through using a common dataset and leave only structural differences. We demonstrate how a systematic model comparison process allows us to pinpoint specific and important structural differences among our models, including specification of technologies as baseload or load following generation, battery state-of-charge at the beginning and end of a modeled period, application of battery roundtrip efficiency, treatment of discount rates, formulation of model end effects, and digit precision of input parameters. Our results show that such a process can be effective for improving consistency across models and building model confidence, substantiating specific modeling choices, reporting uncertainties, and identifying areas for further research and development. We also introduce an open-source test dataset that the modeling community can use for unit testing and build on for benchmarking exercises of more complex models. A community benchmarking effort can increase collaboration among energy modelers and provide transparency regarding the energy transition and energy challenges, for other stakeholders such as policymakers.

电力部门产能扩张模型被学术界、政府和行业研究人员广泛用于政策分析和规划。许多模型在它们的能力、空间和时间分辨率以及研究目的方面存在重叠，但由于参数和结构的差异而产生不同的结果。以前的工作试图确定常用容量扩展模型之间的一些差异，但无法将参数与结构不确定性分开。在这里，我们展示了一个模型基准测试工作，使用高度简化的场景应用于美国电力部门的四个开源模型。我们通过使用通用数据集消除所有参数不确定性，只留下结构差异。我们展示了系统的模型比较过程如何使我们能够确定模型之间的具体和重要的结构差异，包括技术规范，如基本负载或发电后负载、建模周期开始和结束时的电池充电状态、应用电池往返效率、折扣率的处理、模型最终效果的制定以及输入参数的数字精度。我们的结果表明，这样的过程可以有效地提高模型之间的一致性和建立模型置信度，证实特定的建模选择，报告不确定性，并确定进一步研究和开发的领域。我们还介绍了一个开源测试数据集，建模社区可以使用它进行单元测试，并在此基础上构建更复杂模型的基准测试练习。