Operating deep neural networks (DNNs) on devices with limited resources requires the reduction of their memory as well as computational footprint. Popular reduction methods are network quantization or pruning, which either reduce the word length of the network parameters or remove weights from the network if they are not needed. In this article, we discuss a general framework for network reduction which we call Look-Up Table Quantization (LUT-Q). For each layer, we learn a value dictionary and an assignment matrix to represent the network weights. We propose a special solver which combines gradient descent and a one-step k-means update to learn both the value dictionaries and assignment matrices iteratively. This method is very flexible: by constraining the value dictionary, many different reduction problems such as non-uniform network quantization, training of multiplierless networks, network pruning, or simultaneous quantization and pruning can be implemented without changing the solver. This flexibility of the LUT-Q method allows us to use the same method to train networks for different hardware capabilities.

中文翻译：

---

迭代训练网络量化查找表

在资源有限的设备上运行深度神经网络 (DNN) 需要减少其内存和计算量。流行的减少方法是网络量化或剪枝，它们要么减少网络参数的字长，要么在不需要时从网络中删除权重。在本文中，我们讨论了一个用于网络缩减的通用框架，我们称之为查找表量化 (LUT-Q)。对于每一层，我们学习一个值字典和一个分配矩阵来表示网络权重。我们提出了一种特殊的求解器，它结合了梯度下降和一步 k 均值更新，以迭代地学习值字典和分配矩阵。这种方法非常灵活：通过约束值字典，许多不同的归约问题，如非均匀网络量化、无乘子网络的训练、网络剪枝或同时量化和剪枝，都可以在不改变求解器的情况下实现。LUT-Q 方法的这种灵活性使我们能够使用相同的方法针对不同的硬件功能训练网络。