当前位置:
X-MOL 学术
›
arXiv.cs.DB
›
论文详情
Our official English website, www.x-mol.net, welcomes your
feedback! (Note: you will need to create a separate account there.)
Time-Efficient and High-Quality Graph Partitioning for Graph Dynamic Scaling
arXiv - CS - Databases Pub Date : 2021-01-18 , DOI: arxiv-2101.07026 Masatoshi Hanai, Nikos Tziritas, Toyotaro Suzumura, Wentong Cai, Georgios Theodoropoulos
arXiv - CS - Databases Pub Date : 2021-01-18 , DOI: arxiv-2101.07026 Masatoshi Hanai, Nikos Tziritas, Toyotaro Suzumura, Wentong Cai, Georgios Theodoropoulos
The dynamic scaling of distributed computations plays an important role in
the utilization of elastic computational resources, such as the cloud. It
enables the provisioning and de-provisioning of resources to match dynamic
resource availability and demands. In the case of distributed graph processing,
changing the number of the graph partitions while maintaining high partitioning
quality imposes serious computational overheads as typically a time-consuming
graph partitioning algorithm needs to execute each time repartitioning is
required. In this paper, we propose a dynamic scaling method that can
efficiently change the number of graph partitions while keeping its quality
high. Our idea is based on two techniques: preprocessing and very fast edge
partitioning, called graph edge ordering and chunk-based edge partitioning,
respectively. The former converts the graph data into an ordered edge list in
such a way that edges with high locality are closer to each other. The latter
immediately divides the ordered edge list into an arbitrary number of
high-quality partitions. The evaluation with the real-world billion-scale
graphs demonstrates that our proposed approach significantly reduces the
repartitioning time, while the partitioning quality it achieves is on par with
that of the best existing static method.
中文翻译:
用于图动态缩放的省时且高质量的图分区
分布式计算的动态缩放在诸如云之类的弹性计算资源的利用中起着重要作用。它使资源的调配和取消调配能够匹配动态的资源可用性和需求。在分布式图形处理的情况下,在保持高分区质量的同时更改图形分区的数量会带来严重的计算开销,因为通常需要在每次重新分区时执行耗时的图形分区算法。在本文中,我们提出了一种动态缩放方法,该方法可以在保持高质量的同时有效地更改图分区的数量。我们的想法基于两种技术:预处理和非常快速的边缘分区,分别称为图形边缘排序和基于块的边缘分区。前者将图形数据转换为有序边缘列表,以使局部性较高的边缘彼此靠近。后者立即将有序边缘列表划分为任意数量的高质量分区。通过对真实世界的十亿规模图形的评估表明,我们提出的方法可显着减少重新分配时间,同时其实现的划分质量与现有的最佳静态方法相当。
更新日期:2021-01-19
中文翻译:
用于图动态缩放的省时且高质量的图分区
分布式计算的动态缩放在诸如云之类的弹性计算资源的利用中起着重要作用。它使资源的调配和取消调配能够匹配动态的资源可用性和需求。在分布式图形处理的情况下,在保持高分区质量的同时更改图形分区的数量会带来严重的计算开销,因为通常需要在每次重新分区时执行耗时的图形分区算法。在本文中,我们提出了一种动态缩放方法,该方法可以在保持高质量的同时有效地更改图分区的数量。我们的想法基于两种技术:预处理和非常快速的边缘分区,分别称为图形边缘排序和基于块的边缘分区。前者将图形数据转换为有序边缘列表,以使局部性较高的边缘彼此靠近。后者立即将有序边缘列表划分为任意数量的高质量分区。通过对真实世界的十亿规模图形的评估表明,我们提出的方法可显着减少重新分配时间,同时其实现的划分质量与现有的最佳静态方法相当。