This paper proposes a new distributed real-time optimization for MapReduce-style framework in emerging cloud platforms supporting dynamic speed scaling functions. Distinctively different from the existing MapReduce parallelism strategy with fixed specific data chuck sizes, the new approach is able to dynamically dispatch input data of adequate sizes and synthesize interim processing results according to applications and the state of data centers (DCs). The key idea is to decouple the optimizations of data dispatching, processing, and result aggregation without loss of optimality, by employing stochastic optimization techniques. Another important aspect is that we optimize the subproblem of data processing to leverage the energyand speed-configurability of DCs, by optimally deciding the number of servers to be activated at every DC and the CPU speeds of the activated servers. Evident from extensive simulations, the proposed approach is able to increase the throughput-cost ratio by up to 94.3%, as compared to existing initiatives, and substantially improve the throughput in the case of high-rate data streams.

中文翻译：

---

分布式数据中心动态速度扩展的实时优化


本文提出了一种新的分布式实时优化，用于支持动态速度扩展功能的新兴云平台中的 MapReduce 式框架。与现有固定特定数据卡盘大小的 MapReduce 并行策略截然不同，新方法能够根据应用程序和数据中心 (DC) 的状态动态调度足够大小的输入数据并合成临时处理结果。关键思想是通过采用随机优化技术，在不损失最优性的情况下解耦数据调度、处理和结果聚合的优化。另一个重要的方面是，我们通过优化决定每个 DC 上要激活的服务器数量以及激活服务器的 CPU 速度来优化数据处理的子问题，以利用 DC 的能量和速度可配置性。从大量的模拟中可以看出，与现有方案相比，所提出的方法能够将吞吐量成本比提高高达 94.3%，并且在高速率数据流的情况下大幅提高吞吐量。