The Hadoop enabled cloud platforms are gradually becoming preferred computational environment to execute scientific big data workloads in a periodic manner. However, it is observed that the default data placement approach of such cloud platforms is not the efficient one and often ends up with significant data transfer overhead leading to degradation of the overall job completion time. In this article, a Resource and Network-aware Data Placement Algorithm (RENDA) is proposed to reduce the non-local executions and thereby reduce the overall job completion time for periodic workloads in the cloud environment. The entire job execution is modeled as a two-stage execution characterized as data distribution and data processing. The RENDA reduces the time of the stages as mentioned above by estimating the heterogeneous performance of the nodes on a real-time basis followed by careful allocation of data in several installments to participating nodes. The experimental results show that the proposed RENDA algorithm consistently outperforms over the recent state-of-the-art alternatives with as much as 28 percent reduction in data transfer overhead leading to 16 percent reduction in average job completion time with 27 percent average speedup on average job execution.

中文翻译：

---

RENDA：云中定期工作负载的资源和网络感知数据放置算法


支持 Hadoop 的云平台正逐渐成为定期执行科学大数据工作负载的首选计算环境。然而，据观察，此类云平台的默认数据放置方法并不是高效的方法，并且通常会产生大量数据传输开销，从而导致总体作业完成时间缩短。在本文中，提出了一种资源和网络感知数据放置算法（RENDA）来减少非本地执行，从而减少云环境中周期性工作负载的总体作业完成时间。整个作业执行被建模为两阶段执行，其特征是数据分发和数据处理。 RENDA 通过实时估计节点的异构性能，然后将数据分几次仔细分配给参与节点，从而减少了上述阶段的时间。实验结果表明，所提出的 RENDA 算法始终优于最近最先进的替代方案，数据传输开销减少了 28%，导致平均作业完成时间减少了 16%，平均加速提高了 27%作业执行。