Graph neural networks (GNNs) can extract features by learning both the representation of each objects (i.e., graph nodes) as well as the relationship across different objects (i.e., the edges that connect nodes), achieving state-of-the-art performance on a wide range of graph-based tasks. Despite its strengths, utilizing these algorithms in a production environment faces several key challenges as the number of graph nodes and edges amount to several billions to hundreds of billions scale, requiring substantial storage space for training. Unfortunately, existing ML frameworks based on the in-memory processing model significantly hamper the productivity of algorithm developers as it mandates the overall working set to fit within DRAM capacity constraints. In this work, we first study state-of-the-art, large-scale GNN training algorithms. We then conduct a detailed characterization on utilizing capacity-optimized non-volatile memory solutions for storing memory-hungry GNN data, exploring the feasibility of SSDs for large-scale GNN training.

中文翻译：

---

了解非易失性内存对大规模图神经网络训练的影响


图神经网络（GNN）可以通过学习每个对象（即图节点）的表示以及不同对象之间的关系（即连接节点的边）来提取特征，从而实现最先进的性能广泛的基于图形的任务。尽管具有优势，但在生产环境中使用这些算法面临着几个关键挑战，因为图节点和边的数量达到数十亿到数千亿的规模，需要大量的存储空间来进行训练。不幸的是，基于内存处理模型的现有机器学习框架极大地阻碍了算法开发人员的生产力，因为它要求整体工作集适应 DRAM 容量限制。在这项工作中，我们首先研究最先进的大规模 GNN 训练算法。然后，我们对利用容量优化的非易失性存储器解决方案来存储需要大量内存的 GNN 数据进行了详细的表征，探索 SSD 用于大规模 GNN 训练的可行性。