当前位置: X-MOL 学术Hum. Cent. Comput. Inf. Sci. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
TMaR: a two-stage MapReduce scheduler for heterogeneous environments
Human-centric Computing and Information Sciences ( IF 3.9 ) Pub Date : 2020-10-07 , DOI: 10.1186/s13673-020-00247-5
Neda Maleki , Hamid Reza Faragardi , Amir Masoud Rahmani , Mauro Conti , Jay Lofstead

In the context of MapReduce task scheduling, many algorithms mainly focus on the scheduling of Reduce tasks with the assumption that scheduling of Map tasks is already done. However, in the cloud deployments of MapReduce, the input data is located on remote storage which indicates the importance of the scheduling of Map tasks as well. In this paper, we propose a two-stage Map and Reduce task scheduler for heterogeneous environments, called TMaR. TMaR schedules Map and Reduce tasks on the servers that minimize the task finish time in each stage, respectively. We employ a dynamic partition binder for Reduce tasks in the Reduce stage to lighten the shuffling traffic. Indeed, TMaR minimizes the makespan of a batch of tasks in heterogeneous environments while considering the network traffic. The simulation results demonstrate that TMaR outperforms Hadoop-stock and Hadoop-A in terms of makespan and network traffic and achieves by an average of 29%, 36%, and 14% performance using Wordcount, Sort, and Grep benchmarks. Besides, the power reduction of TMaR is up to 12%.



中文翻译:

TMaR:适用于异构环境的两阶段 MapReduce 调度程序

在MapReduce任务调度的背景下,许多算法主要关注Reduce任务的调度,并假设Map任务的调度已经完成。然而,在MapReduce的云部署中,输入数据位于远程存储上,这也表明了Map任务调度的重要性。在本文中,我们提出了一种用于异构环境的两阶段Map和Reduce任务调度器,称为TMaR。 TMaR 在服务器上调度 Map 和 Reduce 任务,分别最小化每个阶段的任务完成时间。我们在Reduce阶段为Reduce任务使用动态分区绑定器来减轻洗牌流量。事实上,TMaR 在考虑网络流量的同时,最大限度地减少了异构环境中一批任务的完成时间。模拟结果表明,TMaR 在完成时间和网络流量方面优于 Hadoop-stock 和 Hadoop-A,并且使用 Wordcount、Sort 和 Grep 基准测试平均实现 29%、36% 和 14% 的性能。此外,TMaR的功耗降低高达12%。

更新日期:2020-10-07
down
wechat
bug