Inference of Convolutional Neural Networks in time critical applications usually requires a GPU. In robotics or embedded devices these are often not available due to energy, space and cost constraints. Furthermore, installation of a deep learning framework or even a native compiler on the target platform is not possible. This paper presents a neural network code generator (NNCG) that generates from a trained CNN a plain ANSI C code file that encapsulates the inference in single a function. It can easily be included in existing projects and due to lack of dependencies, cross compilation is usually possible. Additionally, the code generation is optimized based on the known trained CNN and target platform following four design principles. The system is evaluated utilizing small CNN designed for this application. Compared to TensorFlow XLA and Glow speed-ups of up to 11.81 can be shown and even GPUs are outperformed regarding latency.

中文翻译：

---

用于在资源受限系统上快速推理和简单部署卷积神经网络的 AC 代码生成器

在时间关键型应用中推理卷积神经网络通常需要 GPU。在机器人或嵌入式设备中，由于能源、空间和成本限制，这些通常不可用。此外，不可能在目标平台上安装深度学习框架甚至本机编译器。本文介绍了一种神经网络代码生成器 (NNCG)，它从经过训练的 CNN 生成一个简单的 ANSI C 代码文件，该文件将推理封装在单个函数中。它可以很容易地包含在现有项目中，并且由于缺乏依赖关系，通常可以进行交叉编译。此外，代码生成基于已知的训练有素的 CNN 和目标平台，遵循四个设计原则进行优化。该系统使用专为此应用设计的小型 CNN 进行评估。