当前位置: X-MOL 学术Cluster Comput. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A classification of hadoop job schedulers based on performance optimization approaches
Cluster Computing ( IF 4.4 ) Pub Date : 2021-06-18 , DOI: 10.1007/s10586-021-03339-8
Rana Ghazali , Sahar Adabi , Douglas G. Down , Ali Movaghar

Job scheduling in MapReduce plays a vital role in Hadoop performance. In recent years, many researchers have presented job scheduler algorithms to improve Hadoop performance. Designing a job scheduler that minimizes job execution time with maximum resource utilization is not a straightforward task. The primary purpose of this paper is to investigate agents affecting job scheduler efficiency and present a novel classification for job schedulers based on these factors. We provide a comprehensive overview of existing job schedulers in each group, evaluating their approaches, their effects on Hadoop performance, and comparing their advantages and disadvantages. Finally, we provide recommendations on choosing a preferred job scheduler in different environments for improving Hadoop performance.



中文翻译:

基于性能优化方法的hadoop作业调度器分类

MapReduce 中的作业调度对 Hadoop 性能起着至关重要的作用。近年来,许多研究人员提出了作业调度程序算法来提高 Hadoop 性能。设计一个能够最大限度地减少作业执行时间并最大限度地利用资源的作业调度程序并不是一项简单的任务。本文的主要目的是研究影响作业调度器效率的代理,并基于这些因素提出一种新的作业调度器分类。我们全面概述了每组中现有的作业调度程序,评估了它们的方法、它们对 Hadoop 性能的影响,并比较了它们的优缺点。最后,我们提供了有关在不同环境中选择首选作业调度程序以提高 Hadoop 性能的建议。

更新日期:2021-06-18
down
wechat
bug