In this paper, we find that the conventional convolution operation becomes the bottleneck for extremely efficient binary neural networks (BNNs). To address this issue, we open up a new direction by introducing a reshaped point-wise convolution (RPC) to replace the conventional one to build BNNs. Specifically, we conduct a point-wise convolution after rearranging the spatial information into depth, with which at least 2.25×2.25\times computation reduction can be achieved. Such an efficient RPC allows us to explore more powerful representational capacity of BNNs under a given computation complexity budget. Moreover, we propose to use a balanced activation (BA) to adjust the distribution of the scaled activations after binarization, which enables significant performance improvement of BNNs. After integrating RPC and BA, the proposed network, dubbed as RB-Net, strikes a good trade-off between accuracy and efficiency, achieving superior performance with lower computational cost against the state-of-the-art BNN methods. Specifically, our RB-Net achieves 66.8% Top-1 accuracy with ResNet-18 backbone on ImageNet, exceeding the state-of-the-art Real-to-Binary Net (65.4%) by 1.4% while achieving more than 3×3\times reduction (52M vs. 165M) in computational complexity.

中文翻译：

---

RB-Net：通过重塑逐点卷积和平衡激活来训练高精度和高效的二元神经网络


在本文中，我们发现传统的卷积运算成为极其高效的二元神经网络（BNN）的瓶颈。为了解决这个问题，我们开辟了一个新的方向，引入重塑的逐点卷积（RPC）来取代传统的构建 BNN 的方法。具体来说，我们将空间信息重新排列到深度后进行逐点卷积，这样可以减少至少 2.25×2.25\times 的计算量。这种高效的 RPC 使我们能够在给定的计算复杂度预算下探索 BNN 更强大的表示能力。此外，我们建议使用平衡激活（BA）来调整二值化后缩放激活的分布，这使得 BNN 的性能得到显着提高。在集成 RPC 和 BA 后，所提出的网络（称为 RB-Net）在准确性和效率之间取得了良好的权衡，与最先进的 BNN 方法相比，以更低的计算成本实现了卓越的性能。具体来说，我们的 RB-Net 在 ImageNet 上使用 ResNet-18 主干网络实现了 66.8% 的 Top-1 准确率，比最先进的 Real-to-Binary Net (65.4%) 提高了 1.4%，同时实现了超过 3×3 的准确率\ 计算复杂度减少（52M vs. 165M）。