4-bit and lower precision mobile models are required due to the ever-increasing demand for better energy efficiency in mobile devices. In this work, we report that the activation instability induced by weight quantization (AIWQ) is the key obstacle to sub-4-bit quantization of mobile networks. To alleviate the AIWQ problem, we propose a novel training method called PROgressive-Freezing Iterative Training (PROFIT), which attempts to freeze layers whose weights are affected by the instability problem stronger than the other layers. We also propose a differentiable and unified quantization method (DuQ) and a negative padding idea to support asymmetric activation functions such as h-swish. We evaluate the proposed methods by quantizing MobileNet-v1, v2, and v3 on ImageNet and report that 4-bit quantization offers comparable (within 1.48 % top-1 accuracy) accuracy to full precision baseline. In the ablation study of the 3-bit quantization of MobileNet-v3, our proposed method outperforms the state-of-the-art method by a large margin, 12.86 % of top-1 accuracy.

中文翻译：

---

利润：一种用于亚 4 位 MobileNet 模型的新训练方法

由于对移动设备更高能效的需求不断增长，因此需要 4 位和更低精度的移动模型。在这项工作中，我们报告称权重量化（AIWQ）引起的激活不稳定是移动网络亚 4 位量化的主要障碍。为了缓解 AIWQ 问题，我们提出了一种称为渐进冻结迭代训练 (PROFIT) 的新训练方法，该方法尝试冻结权重受不稳定问题影响的层比其他层更强。我们还提出了一种可微分统一量化方法 (DuQ) 和负填充思想，以支持非对称激活函数，例如 h-swish。我们通过在 ImageNet 上量化 MobileNet-v1、v2 和 v3 来评估所提出的方法，并报告说 4 位量化提供了可比性（在 1. 48 % top-1 准确度）到全精度基线的准确度。在 MobileNet-v3 的 3 位量化的消融研究中，我们提出的方法以 12.86% 的 top-1 准确率大大优于最先进的方法。