Convolutional neural networks (CNNs) have achieved great success in numerous AI applications. To improve inference efficiency of CNNs, researchers have proposed various pruning techniques to reduce both computation intensity and storage overhead. These pruning techniques result in multi-level sparsity irregularities in CNNs. Together with that in activation matrices, which is induced by employment of ReLU activation function, all these sparsity irregularities cause a serious problem of computation resource under-utilization in sparse CNN accelerators. To mitigate this problem, we propose a method of load-balancing based on a workload stealing technique. We demonstrate that this method can be applied to two major inference data-flows, which cover all state-of-the-art sparse CNN accelerators. Based on this method, we present an accelerator, called Crane, which addresses all kinds of sparsity irregularities in CNNs. We perform a fair comparison between Crane and state-of-the-art prior approaches. Experimental results show that Crane improves performance by $27\%\sim 88\%$27%∼88% and reduces energy consumption by $16\%\sim 48\%$16%∼48%, respectively, compared to the counterparts.

中文翻译：

---

Crane：减轻由 CNN 中的稀疏不规则性引起的加速器未充分利用

卷积神经网络 (CNN) 在众多 AI 应用中取得了巨大成功。为了提高 CNN 的推理效率，研究人员提出了各种修剪技术来降低计算强度和存储开销。这些修剪技术导致 CNN 中出现多级稀疏不规则性。连同由使用 ReLU 激活函数引起的激活矩阵中的情况，所有这些稀疏不规则性导致稀疏 CNN 加速器中计算资源未充分利用的严重问题。为了缓解这个问题，我们提出了一种基于工作负载窃取技术的负载平衡方法。我们证明了这种方法可以应用于两个主要的推理数据流，它们涵盖了所有最先进的稀疏 CNN 加速器。基于这种方法，我们提出了一种加速器，称为 Crane，它解决了 CNN 中的各种稀疏不规则性问题。我们对 Crane 和最先进的现有方法进行了公平的比较。实验结果表明，Crane 提高了性能$27\%\sim 88\%$27%～88% 并通过以下方式减少能源消耗 $16\%\sim 48\%$16%～48%，分别与同行相比。