Disk I/O is the major performance bottleneck of existing out-of-core graph processing systems. We found that the total I/O amount can be reduced by loading more vertices into memory every time. Although task partitioning of a graph processing system is traditionally considered equivalent to the graph partition problem, this assumption is untrue for many Machine Learning and Data Mining (MLDM) problems: instead of a single value, a vector of data elements is defined as the property for each vertex/edge. By dividing each vertex into multiple sub-vertices, more vertices can be loaded into memory every time, leading to less amount of disk I/O. To explore this new opportunity, we propose a category of 3-D partitioning algorithm that considers the hidden dimension to partition the property vector. The 3-D partitioning algorithm provides a new tradeoff to reduce communication costs, which is adaptive to both distributed and out-of-core scenarios. Based on it, we build a distributed graph processing system Cube and an out-of-core system SingleCube. Since network traffic is significantly reduced, Cube outperforms state-of-the-art graph-parallel system PowerLyra by up to $4.7\times$4.7×. By largely reducing the disk I/O amount, the performance of SingleCube is significantly better than state-of-the-art out-of-core system GridGraph (up to $4.5\times$4.5×).

中文翻译：

---

用于大规模图形处理的 3-D 分区

磁盘 I/O 是现有的核外图形处理系统的主要性能瓶颈。我们发现每次将更多的顶点加载到内存中可以减少总 I/O 量。尽管传统上认为图处理系统的任务分区等效于图分区问题，但对于许多机器学习和数据挖掘 (MLDM) 问题，这种假设是不正确的：不是单个值，而是一个向量数据元素的数量被定义为每个顶点/边的属性。通过将每个顶点划分为多个子顶点，每次可以将更多的顶点加载到内存中，从而减少磁盘 I/O 量。为了探索这个新机会，我们提出了一个类别3-D 分区考虑隐藏维度来划分属性向量的算法。3-D 分区算法提供了一种新的权衡来降低通信成本，它适用于分布式和核外场景。基于它，我们构建了一个分布式图处理系统立方体 和一个核外系统 单立方. 由于网络流量显着减少，立方体 性能优于最先进的图形并行系统 PowerLyra 高达 $4.7\times$4.7×. 通过大幅减少磁盘 I/O 量，单立方 明显优于最先进的核外系统 GridGraph（高达 $4.5\times$4.5×）。