Many applications in important problem domains such as machine learning and computer vision are streaming applications that take a sequence of inputs over time. It is challenging to find knob settings that optimize the run-time performance of such applications because the optimal knob settings are usually functions of inputs, computing platforms, time as well as user's requirements, which can be very diverse. Most prior works address this problem by offline profiling followed by training models for control. However, profiling-based approaches incur large overhead before execution; it is also difficult to redeploy them in other run-time configurations. In this paper, we propose Sonic, a sampling-based online controller for long-running streaming applications that does not require profiling ahead of time. Within each phase of a streaming application's execution, Sonic utilizes the beginning portion to sample the knob space strategically and aims to pick the optimal knob setting for the rest of the phase, given a user-specified constrained optimization problem. A hybrid approach of machine learning regressions and Bayesian optimization are used for better overall sampling choices. Sonic is implemented independent of application, device, input, performance objective and constraints. We evaluate Sonic on traditional parallel benchmarks as well as on deep learning inference benchmarks across multiple platforms. Our experiments show that when using Sonic to control knob settings, application run-time performance is only 5.3% less than if optimal knob settings were used, demonstrating that Sonic is able to find near-optimal knob settings under diverse run-time configurations without prior knowledge quickly.

中文翻译：

---

Sonic：用于流媒体应用程序的基于采样的在线控制器

机器学习和计算机视觉等重要问题领域中的许多应用程序都是流应用程序，它们随着时间的推移接受一系列输入。找到优化此类应用程序运行时性能的旋钮设置具有挑战性，因为最佳旋钮设置通常是输入、计算平台、时间以及用户需求的函数，这些功能可能非常多样化。大多数先前的工作通过离线分析和训练控制模型来解决这个问题。然而，基于分析的方法在执行前会产生大量开销；在其他运行时配置中重新部署它们也很困难。在本文中，我们提出了 Sonic，这是一种基于采样的在线控制器，用于不需要提前分析的长时间运行的流应用程序。在流应用程序执行的每个阶段，Sonic 利用开始部分策略性地对旋钮空间进行采样，并在给定用户指定的约束优化问题的情况下，旨在为阶段的其余部分选择最佳旋钮设置。机器学习回归和贝叶斯优化的混合方法用于更好的整体采样选择。Sonic 的实现独立于应用程序、设备、输入、性能目标和约束。我们在传统的并行基准以及跨多个平台的深度学习推理基准上评估 Sonic。我们的实验表明，当使用 Sonic 控制旋钮设置时，应用程序运行时性能仅比使用最佳旋钮设置时低 5.3%，