当前位置: X-MOL 学术J. Netw. Comput. Appl. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
SPO: A Secure and Performance-aware Optimization for MapReduce Scheduling
Journal of Network and Computer Applications ( IF 7.7 ) Pub Date : 2020-12-01 , DOI: 10.1016/j.jnca.2020.102944
Neda Maleki , Amir Masoud Rahmani , Mauro Conti

MapReduce is a common framework that effectively processes multi-petabyte data in a distributed manner. Therefore, MapReduce is widely used in heterogeneous environments, such as cloud, to provide performance adequate for system needs. Despite the MapReduce benefits, tweaking the system configuration to achieve the maximum performance is still challenging and needs deep expertise. Besides, some new MapReduce security issues, which has not been well-addressed yet, are recently raised. In this paper, we present a performance-aware and secure framework, named SPO, to minimize the makespan of the tasks while considering task security constraints. Inspired by the HEFT algorithm, first, we introduce SPO, which proposes a two-stage static scheduler in Map and Reduce phases, respectively, to minimize makespan while considering network traffic. Plus, SPO introduces a mathematical optimization model of the proposed scheduler aiming to estimate the system performance while considering security constraints with an error of less than 2%. The experimental results demonstrate that SPO outperforms Hadoop-stock in terms of makespan and network traffic by 29% and 31%, respectively, for the tasks running in heterogeneous environments.



中文翻译:

SPO:MapReduce调度的安全和性能感知优化

MapReduce是一个通用框架,可以有效地以分布式方式处理数PB的数据。因此,MapReduce广泛用于异构环境(例如云​​)中,以提供足以满足系统需求的性能。尽管有MapReduce的好处,但是调整系统配置以实现最佳性能仍然是一项挑战,需要深厚的专业知识。此外,最近还提出了一些尚未解决的新MapReduce安全问题。在本文中,我们提出了一个性能感知和安全的框架,名为小号PØ,以在考虑任务安全性约束的同时最大程度地减少任务的工期。受启发HËFŤ 算法,首先,我们介绍 小号PØ,分别在Map和Reduce阶段提出了一个两阶段的静态调度程序,以在考虑网络流量的同时最大程度地缩短制造时间。加,小号PØ引入了拟议调度程序的数学优化模型,旨在在考虑安全约束且误差小于2%的同时估算系统性能。实验结果表明小号PØ 就异构环境中运行的任务而言,其生成量和网络流量分别比Hadoop存量高29%和31%。

更新日期:2020-12-10
down
wechat
bug