Most online service providers deploy their own data stream processing systems in the cloud to conduct large-scale and real-time data analytics. However, such systems, e.g., Apache Heron, often adopt naive scheduling schemes to distribute data streams (in the units of tuples) among processing instances, which may result in workload imbalance and system disruption. Hence, there still exists a mismatch between the temporal variations of data streams and such inflexible scheduling scheme designs. Besides, the fundamental limits of benefits of predictive scheduling to data stream processing systems remain unexplored. In this article, we focus on the problem of tuple scheduling with predictive service in Apache Heron. With a careful choice in the granularity of system modeling and decision making, we formulate the problem as a stochastic network optimization problem and propose POTUS , an online predictive scheduling scheme that aims to minimize the response time of data stream processing by steering data streams in a distributed fashion. Theoretical analysis and simulation results show that POTUS achieves an ultra-low response time with a stability guarantee. Moreover, POTUS only requires mild-value of future information to effectively reduce the response time, even with mis-prediction.

中文翻译：

---

POTUS：数据流处理系统的预测在线元组调度

大多数在线服务提供商都在云端部署了自己的数据流处理系统，以进行大规模、实时的数据分析。然而，此类系统，例如 Apache Heron，通常采用朴素的调度方案在处理实例之间分配数据流（以元组为单位），这可能导致工作负载不平衡和系统中断。因此，数据流的时间变化与这种不灵活的调度方案设计之间仍然存在不匹配。此外，预测调度对数据流处理系统的好处的基本限制仍未探索。在本文中，我们将重点关注 Apache Heron 中具有预测服务的元组调度问题。通过仔细选择系统建模和决策的粒度，POTUS ，一种在线预测调度方案，旨在通过以分布式方式引导数据流来最小化数据流处理的响应时间。理论分析和仿真结果表明，POTUS 实现了超低的响应时间，并保证了稳定性。此外，POTUS 只需要未来信息的温和价值即可有效减少响应时间，即使出现错误预测也是如此。