Modern deep neural network (DNN) training jobs use complex and heterogeneous software/hardware stacks. The efficacy of software-level optimizations can vary significantly when used in different deployment configurations. It is onerous and error-prone for ML practitioners and system developers to implement each optimization separately, and determine which ones will improve performance in their own configurations. Unfortunately, existing profiling tools do not aim to answer predictive questions such as "How will optimization X affect the performance of my model?". We address this critical limitation, and proposes a new profiling tool, Daydream, to help programmers efficiently explore the efficacy of DNN optimizations. Daydream models DNN execution with a fine-grained dependency graph based on low-level traces collected by CUPTI, and predicts runtime by simulating execution based on the dependency graph. Daydream maps the low-level traces using DNN domain-specific knowledge, and introduces a set of graph-transformation primitives that can easily model a wide variety of optimizations. We show that Daydream is able to model most mainstream DNN optimization techniques, and accurately predict the efficacy of optimizations that will result in significant performance improvements.

中文翻译：

---

Daydream：准确估计 DNN 训练优化的效果

现代深度神经网络 (DNN) 训练作业使用复杂的异构软件/硬件堆栈。在不同的部署配置中使用时，软件级优化的功效可能会有很大差异。对于 ML 从业者和系统开发人员来说，分别实施每个优化并确定哪些优化将在他们自己的配置中提高性能是繁重且容易出错的。不幸的是，现有的分析工具并不旨在回答诸如“优化 X 将如何影响我的模型的性能？”等预测性问题。我们解决了这一关键限制，并提出了一种新的分析工具 Daydream，以帮助程序员有效地探索 DNN 优化的功效。Daydream 使用基于 CUPTI 收集的低级跟踪的细粒度依赖图对 DNN 执行进行建模，并通过基于依赖图模拟执行来预测运行时间。Daydream 使用 DNN 领域特定的知识映射低级跟踪，并引入了一组图转换原语，可以轻松地对各种优化进行建模。我们表明 Daydream 能够对大多数主流 DNN 优化技术进行建模，并准确预测优化的效果，从而显着提高性能。