Understanding and predicting the performance of big data applications running in the cloud or on-premises could help minimise the overall cost of operations and provide opportunities in efforts to identify performance bottlenecks. The complexity of the low-level internals of big data frameworks and the ubiquity of application and workload configuration parameters makes it challenging and expensive to come up with comprehensive performance modelling solutions. In this paper, instead of focusing on a wide range of configurable parameters, we studied the low-level internals of the MapReduce communication pattern and used a minimal set of performance drivers to develop a set of phase level parametric models for approximating the execution time of a given application on a given cluster. Models can be used to infer the performance of unseen applications and approximate their performance when an arbitrary dataset is used as input. Our approach is validated by running empirical experiments in two setups. On average the error rate in both setups is plus or minus 10% from the measured values.

中文翻译：

---

MapReduce 通信模式的基准测试和性能建模

了解和预测在云中或本地运行的大数据应用程序的性能有助于最大限度地降低总体运营成本，并为识别性能瓶颈提供机会。大数据框架底层内部结构的复杂性以及无处不在的应用程序和工作负载配置参数使得提出全面的性能建模解决方案具有挑战性且成本高昂。在本文中，我们没有关注大范围的可配置参数，而是研究了 MapReduce 通信模式的低级内部结构，并使用了一组最小的性能驱动程序来开发一组阶段级参数模型，用于逼近给定集群上的给定应用程序。模型可用于推断未知应用程序的性能，并在将任意数据集用作输入时近似其性能。我们的方法通过在两种设置中运行经验实验得到验证。平均而言，两种设置中的错误率与测量值相差正负 10%。