Tensor decomposition is a promising approach for low-power and real-time application of neural networks on resource-constrained embedded devices. This paper proposes AutoRank, an end-to-end framework for customizing neural network decomposition using cross-layer rank-selection. For many-layer networks, determining the optimal decomposition ranks is a cumbersome task. To overcome this challenge, we establish a state-action-reward system that effectively absorbs inference accuracy and platform specifications into the rank-selection policy. Our proposed framework brings platform characteristics and performance in the customization loop to enable direct incorporation of hardware cost, e.g., runtime and memory footprint. By means of this hardware-awareness, AutoRank customization engine delivers high accuracy decomposed deep neural networks with low execution cost. Our framework minimizes the engineering cost associated with rank selection by providing an automated API for AutoRank that is compatible with popular deep learning libraries and can be readily used by developers.

中文翻译：

---

AutoRank：自动排名选择以实现有效的神经网络定制


张量分解是一种在资源受限的嵌入式设备上低功耗、实时应用神经网络的有前景的方法。本文提出了 AutoRank，一种使用跨层排名选择定制神经网络分解的端到端框架。对于多层网络来说，确定最佳分解秩是一项繁琐的任务。为了克服这一挑战，我们建立了一个状态-行动-奖励系统，有效地将推理准确性和平台规范吸收到排名选择策略中。我们提出的框架将平台特性和性能引入定制循环中，以实现直接合并硬件成本，例如运行时和内存占用。通过这种硬件感知，AutoRank 定制引擎以较低的执行成本提供高精度的分解深度神经网络。我们的框架通过为 AutoRank 提供与流行的深度学习库兼容且易于开发人员使用的自动化 API，最大限度地降低了与排名选择相关的工程成本。