Graph neural networks (GNNs) have received intense interest as a rapidly expanding class of machine learning models remarkably well-suited for materials applications. To date, a number of successful GNNs have been proposed and demonstrated for systems ranging from crystal stability to electronic property prediction and to surface chemistry and heterogeneous catalysis. However, a consistent benchmark of these models remains lacking, hindering the development and consistent evaluation of new models in the materials field. Here, we present a workflow and testing platform, MatDeepLearn, for quickly and reproducibly assessing and comparing GNNs and other machine learning models. We use this platform to optimize and evaluate a selection of top performing GNNs on several representative datasets in computational materials chemistry. From our investigations we note the importance of hyperparameter selection and find roughly similar performances for the top models once optimized. We identify several strengths in GNNs over conventional models in cases with compositionally diverse datasets and in its overall flexibility with respect to inputs, due to learned rather than defined representations. Meanwhile several weaknesses of GNNs are also observed including high data requirements, and suggestions for further improvement for applications in materials chemistry are proposed.

中文翻译：

---

材料化学基准图神经网络

随着快速扩展的机器学习模型类别非常适合材料应用，图神经网络（GNN）引起了人们的极大兴趣。迄今为止，已经提出并证明了许多成功的GNN，其应用范围从晶体稳定性到电子性能预测，再到表面化学和多相催化。但是，仍然缺乏这些模型的一致基准，这阻碍了材料领域新模型的开发和一致评估。在这里，我们提供了一个工作流程和测试平台MatDeepLearn，用于快速，可重复地评估和比较GNN和其他机器学习模型。我们使用该平台来优化和评估计算材料化学中几个代表性数据集上表现最好的GNN的选择。从我们的调查中，我们注意到选择超参数的重要性，并发现优化后的顶级模型的性能大致相似。在具有组成多样的数据集的情况下，由于学习到的而不是定义的表示，我们确定了GNN优于常规模型的几种优势。同时还观察到了GNNs的一些弱点，包括对数据的高要求，并提出了进一步改进材料化学应用的建议。由于学习而不是定义表示。同时还观察到了GNNs的一些弱点，包括对数据的高要求，并提出了进一步改进材料化学应用的建议。由于学习而不是定义表示。同时还观察到了GNNs的一些弱点，包括对数据的高要求，并提出了进一步改进材料化学应用的建议。