With the rapid growth of deep learning models and deep learning-based applications, how to accelerate the inference of deep neural networks, especially neural network operators, has become an increasingly important research area. As a bridge between a front-end deep learning framework and a back-end hardware platform, deep learning compilers aim to optimize various deep learning models for a range of hardware platforms with model- and hardware-specific optimizations. Apache TVM (or TVM for short), a well-known open-source deep learning compiler, uses a customized domain-specific language, called Tensor Expression Language, to define hardware-specific optimizations for neural network operators. TVM also allows users to write tensor expressions to design customized optimizations for specific operators. However, TVM does not assist users with supporting information, such as what computations are performed within an operator, and tools for optimizing the operators in a deep learning model. In addition, tensor expressions have an entirely different syntax from imperative languages and are not easy to get started with. Furthermore, although TVM comes with an auto-tuning module, called AutoTVM, which facilitates the tuning of optimization configurations (e.g., tiling size and loop order), AutoTVM takes quite a long time to search the optimum configurations for a set of optimizations. In this paper, we present DLOOPT, an optimization assistant that assists optimization developers in designing effective optimizations for neural network operators and/or obtaining optimum optimization configurations in a timely manner. DLOOPT specifically addresses three key aspects: (1) developers can focus only on designing optimizations by using DLOOPT, which offers sufficient information about the operators of a given model and provides an easier way to write optimizations, (2) the number of optimizations that developers need to design can be minimized by using DLOOPT, which allows optimizations to be reused, and (3) the tuning process can be greatly simplified by using DLOOPT, which implements a set of tuning strategies in AutoTVM. The evaluation results showed that DLOOPT reduced more than 99% of time in terms of developing adequate optimizations for operators in a model. We believe that DLOOPT is friendly to optimization developers and allows them to quickly develop effective optimizations for neural network operators.

随着深度学习模型和基于深度学习的应用的快速增长，如何加速深度神经网络，尤其是神经网络算子的推理，已成为越来越重要的研究领域。作为前端深度学习框架和后端硬件平台之间的桥梁，深度学习编译器旨在通过模型和硬件特定的优化为一系列硬件平台优化各种深度学习模型。Apache TVM（或简称 TVM）是一种著名的开源深度学习编译器，它使用一种定制的特定领域语言，称为张量表达式语言，为神经网络算子定义特定于硬件的优化。TVM 还允许用户编写张量表达式来为特定的算子设计定制的优化。然而，TVM 不会为用户提供支持信息，例如在算子中执行了哪些计算，以及在深度学习模型中优化算子的工具。此外，张量表达式的语法与命令式语言完全不同，不容易上手。此外，尽管 TVM 带有一个称为 AutoTVM 的自动调整模块，它有助于调整优化配置（例如，平铺大小和循环顺序），但 AutoTVM 需要相当长的时间来搜索一组优化的最佳配置。在本文中，我们提出 张量表达式的语法与命令式语言完全不同，并且不容易上手。此外，尽管 TVM 带有一个称为 AutoTVM 的自动调整模块，它有助于调整优化配置（例如，平铺大小和循环顺序），但 AutoTVM 需要相当长的时间来搜索一组优化的最佳配置。在本文中，我们提出 张量表达式的语法与命令式语言完全不同，并且不容易上手。此外，尽管 TVM 带有一个称为 AutoTVM 的自动调整模块，它有助于调整优化配置（例如，平铺大小和循环顺序），但 AutoTVM 需要相当长的时间来搜索一组优化的最佳配置。在本文中，我们提出DLOOPT，一个优化助手，帮助优化开发人员为神经网络算子设计有效的优化和/或及时获得最佳优化配置。DLOOPT 具体解决三个关键方面：（1）开发人员可以只专注于使用 DLOOPT 设计优化，它提供了有关给定模型的运算符的足够信息并提供了编写优化的更简单方法，（2）开发人员的优化数量使用 DLOOPT 可以最大限度地减少设计需要，允许重用优化，并且 (3) 使用 DLOOPT 可以大大简化调优过程，它在 AutoTVM 中实现了一组调优策略。评估结果表明，DLOOPT 在为模型中的算子开发适当优化方面减少了 99% 以上的时间。