Recent years have witnessed a great advance of deep learning in a variety of vision tasks. Many state-of-the-art deep neural networks suffer from large size and high complexity, which makes them difficult to deploy in resource-limited platforms such as mobile devices. To this end, low-precision neural networks are widely studied that quantize weights or activations into the low-bit format. Although efficient, low-precision networks are usually difficult to train and encounter severe accuracy degradation. In this paper, we propose a new training strategy based on progressive learning for image classification. First, we equip each low-precision convolutional layer with an ancillary full-precision convolutional layer based on a low-precision network structure. Second, a decay method is introduced to reduce the output of the added full-precision convolution gradually, which keeps the resulting topology structure the same as the original low-precision convolution. Extensive experiments on SVHN, CIFAR and ILSVRC-2012 datasets reveal that the proposed method can bring faster convergence and higher accuracy for low-precision neural networks.

中文翻译：

---

用于图像分类的低精度网络的渐进式学习

近年来，深度学习在各种视觉任务中取得了巨大进步。许多最先进的深度神经网络尺寸大、复杂度高，这使得它们难以部署在资源有限的平台（如移动设备）中。为此，广泛研究了将权重或激活量化为低位格式的低精度神经网络。尽管高效、低精度的网络通常难以训练并且会遇到严重的精度下降。在本文中，我们提出了一种基于渐进学习的图像分类新训练策略。首先，我们为每个低精度卷积层配备一个基于低精度网络结构的辅助全精度卷积层。第二，引入衰减方法，逐渐减少添加的全精度卷积的输出，从而保持所得拓扑结构与原始低精度卷积相同。在 SVHN、CIFAR 和 ILSVRC-2012 数据集上的大量实验表明，所提出的方法可以为低精度神经网络带来更快的收敛和更高的精度。