当前位置: X-MOL 学术Computing › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Dynamic energy-efficient scheduling for streaming applications in storm
Computing ( IF 3.7 ) Pub Date : 2021-06-08 , DOI: 10.1007/s00607-021-00961-7
Hongjian Li , Hongxi Dai , Zengyan Liu , Hao Fu , Yang Zou

With the rapid development of information technology, the data generated by the Internet has exploded in recent years. The proliferation of data has brought about a huge increase in the energy consumption for data processing especially in real-time processing framework for big data. In this study, two energy-efficient scheduling algorithms are proposed to reduce energy consumption for streaming applications in Storm. First, an energy consumption model is designed for Storm framework. Then this model is introduced into Storm by an energy consumption monitoring module. For proposed algorithm 1, the energy consumption of the processing tasks is minimized by integrating the tasks into the low energy consumption nodes. For proposed algorithm 2, load balance and energy consumption of Storm cluster are traded off and optimized by sorting the Slot utilization of low energy consumption nodes in the cluster and assigning tasks priority to the low Slot utilization nodes. Test on Hibench workload, the proposed algorithms reduce the total energy consumption of Storm cluster up to 32% compared with the traditional scheduling algorithms. It shows that the proposed scheduling algorithms can effectively reduce the total energy consumption of the Storm cluster while satisfying the deadline constrains.



中文翻译:

风暴中流应用的动态节能调度

近年来,随着信息技术的飞速发展,互联网产生的数据呈爆炸式增长。数据的激增带来了数据处理的能耗大幅增加,尤其是在大数据的实时处理框架中。在这项研究中,提出了两种节能调度算法来降低 Storm 中流应用程序的能耗。首先,针对Storm框架设计了能耗模型。然后这个模型被一个能耗监控模块引入到 Storm 中。对于所提出的算法 1,通过将任务集成到低能耗节点来最小化处理任务的能耗。对于提议的算法 2,Storm集群的负载均衡和能耗通过对集群中低能耗节点的Slot利用率进行排序,并为低Slot利用率节点分配任务优先级来进行权衡和优化。在 Hibench 工作负载上进行测试,与传统调度算法相比,所提出的算法将 Storm 集群的总能耗降低了 32%。结果表明,所提出的调度算法在满足截止时间约束的同时,可以有效降低Storm集群的总能耗。

更新日期:2021-06-08
down
wechat
bug