The workflows of the predominant datacenter services are underlaid by various Fork-Join structures. Due to the lack of good understanding of the performance of Fork-Join structures in general, today's datacenters often operate under low resource utilization to meet stringent service level objectives (SLOs), e.g., in terms of tail and/or mean latency, for such services. Hence, to achieve high resource utilization, while meeting stringent SLOs, it is of paramount importance to be able to accurately predict the tail and/or mean latency for a broad range of Fork-Join structures of practical interests. In this article, we propose a black-box Fork-Join model that covers a wide range of Fork-Join structures for the prediction of tail and mean latency, called ForkTail and ForkMean, respectively. We derive highly computational effective, empirical expressions for tail and mean latency as functions of means and variances of task response times. Our extensive testing results based on model-based and trace-driven simulations, as well as a real-world case study in a cloud environment demonstrate that the models can consistently predict the tail and mean latency within 20 and 15 percent prediction errors at 80 and 90 percent load levels, respectively, for heavy-tailed workloads, and at any load levels for light-tailed workloads. Moreover, our sensitivity analysis demonstrates that such errors can be well compensated for with no more than 7 percent resource overprovisioning. Consequently, the proposed prediction model can be used as a powerful tool to aid the design of tail-and-mean-latency guaranteed job scheduling and resource provisioning, especially at high load, for datacenter applications.

中文翻译：

---

适用于数据密集型应用程序的黑盒 Fork-Join 延迟预测模型


主要数据中心服务的工作流程以各种 Fork-Join 结构为基础。由于对 Fork-Join 结构的总体性能缺乏良好的理解，当今的数据中心通常在低资源利用率下运行，以满足严格的服务级别目标 (SLO)，例如，在尾部和/或平均延迟方面，对于此类服务。因此，为了实现高资源利用率，同时满足严格的 SLO，能够准确预测具有实际意义的各种 Fork-Join 结构的尾部和/或平均延迟至关重要。在本文中，我们提出了一种黑盒 Fork-Join 模型，涵盖了广泛的 Fork-Join 结构，用于预测尾延迟和平均延迟，分别称为 ForkTail 和 ForkMean。我们得出了尾部延迟和平均延迟的高度计算有效的经验表达式，作为任务响应时间的均值和方差的函数。我们基于基于模型和跟踪驱动的模拟的广泛测试结果，以及云环境中的真实案例研究表明，这些模型可以一致地预测尾部延迟和平均延迟，在 80 和 15% 的预测误差范围内。对于重尾工作负载，分别为 90% 的负载水平，对于轻尾工作负载，分别为任意负载水平。此外，我们的敏感性分析表明，此类错误可以通过不超过 7% 的资源过度配置得到很好的补偿。因此，所提出的预测模型可以用作强大的工具，帮助设计数据中心应用程序的尾部和平均延迟保证的作业调度和资源配置，特别是在高负载下。