The ever-growing computational demands of increasingly complex machine learning models frequently necessitate the use of powerful cloud-based infrastructure for their training. Binary neural networks are known to be promising candidates for on-device inference due to their extreme compute and memory savings over higher-precision alternatives. In this paper, we demonstrate that they are also strongly robust to gradient quantization, thereby making the training of modern models on the edge a practical reality. We introduce a low-cost binary neural network training strategy exhibiting sizable memory footprint reductions and energy savings vs Courbariaux & Bengio's standard approach. Against the latter, we see coincident memory requirement and energy consumption drops of 2--6$\times$, while reaching similar test accuracy in comparable time, across a range of small-scale models trained to classify popular datasets. We also showcase ImageNet training of ResNetE-18, achieving a 3.12$\times$ memory reduction over the aforementioned standard. Such savings will allow for unnecessary cloud offloading to be avoided, reducing latency, increasing energy efficiency and safeguarding privacy.

中文翻译：

---

在边缘启用二进制神经网络训练

日益复杂的机器学习模型对计算的需求不断增长，因此经常需要使用功能强大的基于云的基础架构进行培训。已知二进制神经网络是设备上推理的有前途的候选者，因为它们比高精度替代方案具有极大的计算量和节省的存储空间。在本文中，我们证明了它们对梯度量化也具有很强的鲁棒性，从而使在边缘上训练现代模型成为现实。与Courbariaux和Bengio的标准方法相比，我们引入了一种低成本的二进制神经网络训练策略，该策略具有显着的内存占用减少和节能效果。相对于后者，我们看到同时的内存需求和能耗下降了2--6 $ \ times $，同时在可比较的时间内达到了类似的测试精度，一系列经过训练以对流行数据集进行分类的小规模模型。我们还展示了ResNetE-18的ImageNet培训，与上述标准相比，内存减少了3.12 $ \ times $。这样的节省将可以避免不必要的云卸载，减少延迟，提高能效并保护隐私。