当前位置: X-MOL 学术arXiv.cs.DB › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Time-Efficient and High-Quality Graph Partitioning for Graph Dynamic Scaling
arXiv - CS - Databases Pub Date : 2021-01-18 , DOI: arxiv-2101.07026
Masatoshi Hanai, Nikos Tziritas, Toyotaro Suzumura, Wentong Cai, Georgios Theodoropoulos

The dynamic scaling of distributed computations plays an important role in the utilization of elastic computational resources, such as the cloud. It enables the provisioning and de-provisioning of resources to match dynamic resource availability and demands. In the case of distributed graph processing, changing the number of the graph partitions while maintaining high partitioning quality imposes serious computational overheads as typically a time-consuming graph partitioning algorithm needs to execute each time repartitioning is required. In this paper, we propose a dynamic scaling method that can efficiently change the number of graph partitions while keeping its quality high. Our idea is based on two techniques: preprocessing and very fast edge partitioning, called graph edge ordering and chunk-based edge partitioning, respectively. The former converts the graph data into an ordered edge list in such a way that edges with high locality are closer to each other. The latter immediately divides the ordered edge list into an arbitrary number of high-quality partitions. The evaluation with the real-world billion-scale graphs demonstrates that our proposed approach significantly reduces the repartitioning time, while the partitioning quality it achieves is on par with that of the best existing static method.

中文翻译:

用于图动态缩放的省时且高质量的图分区

分布式计算的动态缩放在诸如云之类的弹性计算资源的利用中起着重要作用。它使资源的调配和取消调配能够匹配动态的资源可用性和需求。在分布式图形处理的情况下,在保持高分区质量的同时更改图形分区的数量会带来严重的计算开销,因为通常需要在每次重新分区时执行耗时的图形分区算法。在本文中,我们提出了一种动态缩放方法,该方法可以在保持高质量的同时有效地更改图分区的数量。我们的想法基于两种技术:预处理和非常快速的边缘分区,分别称为图形边缘排序和基于块的边缘分区。前者将图形数据转换为有序边缘列表,以使局部性较高的边缘彼此靠近。后者立即将有序边缘列表划分为任意数量的高质量分区。通过对真实世界的十亿规模图形的评估表明,我们提出的方法可显着减少重新分配时间,同时其实现的划分质量与现有的最佳静态方法相当。
更新日期:2021-01-19
down
wechat
bug