当前位置: X-MOL 学术Comput. Netw. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Cluster Load based Content Distribution and Speculative Execution for Geographically Distributed Cloud Environment
Computer Networks ( IF 5.6 ) Pub Date : 2021-01-03 , DOI: 10.1016/j.comnet.2021.107807
Chunlin Li , Mingyang Song , Qingchuan Zhang , Youlong Luo

The scale of big data has shown an explosive growth, which makes the processing of big data put forward higher requirements on data centers, and a single data center can no longer meet the needs of big data processing. To deal with this situation, a geographically distributed cloud system needs to be built. However, in the geographically distributed cloud system, each data center is distributed in different geographic locations, which makes the data placement operations in the geographically distributed cloud system lead to greater overhead. To solve this problem, this paper proposes a data placement strategy. This strategy comprehensively considers the data transmission latency, bandwidth cost, cloud server storage capacity, and load capacity during the data placement process, and formulates a data placement problem that minimizes the energy consumption of data transmission. Then the minimum set cover method based on Lagrangian relaxation is used to solve this problem and obtain the optimal data placement scheme. On the other hand, in a geographically distributed cloud data center, the execution progress of the job submitted by the user will be affected by the straggler task. To solve this problem, this paper proposes a speculative execution strategy for the geographically distributed cloud system. This strategy performs different speculative execution operations according to the state of the cluster load, and then calculates the load capacity of the nodes in the cluster. The node with the strongest load capacity in the cluster is used to perform speculative execution operations. Experimental results show that the proposed data placement strategy can effectively improve the performance of the energy consumption, the data storage cost, the network transmission cost and the data transmission time. The proposed speculative execution strategy can effectively improve the performance of the job completion time, cluster throughput and QoS satisfaction rate.



中文翻译:

基于集群负载的内容分布和地理执行的云环境的推测执行

大数据规模呈现爆炸性增长,这使得大数据处理对数据中心提出了更高的要求,单个数据中心已不能满足大数据处理的需求。为了应对这种情况,需要构建一个地理上分散的云系统。但是,在地理上分布的云系统中,每个数据中心分布在不同的地理位置,这使得在地理上分布的云系统中的数据放置操作导致更大的开销。为了解决这个问题,本文提出了一种数据放置策略。该策略综合考虑了数据放置过程中的数据传输延迟,带宽成本,云服务器存储容量和负载容量,并提出一个数据放置问题,以最大程度地减少数据传输的能耗。然后使用基于拉格朗日松弛的最小集覆盖方法来解决该问题并获得最佳的数据放置方案。另一方面,在地理上分散的云数据中心中,用户提交的作业的执行进度将受到散乱任务的影响。为了解决这个问题,本文提出了一种针对地理分布的云系统的推测执行策略。该策略根据集群负载的状态执行不同的推测执行操作,然后计算集群中节点的负载容量。集群中负载能力最强的节点用于执行推测执行操作。实验结果表明,提出的数据放置策略可以有效提高能耗,数据存储成本,网络传输成本和数据传输时间的性能。所提出的推测执行策略可以有效地提高作业完成时间,集群吞吐量和QoS满意度的性能。

更新日期:2021-01-04
down
wechat
bug