Filter pruning for a pre-trained convolutional neural network is most normally performed through human-made constraints or criteria such as norms, ranks, etc. Typically, the pruning pipeline comprises two-stage: first learn a sparse structure from the original model, then optimize the weights in the new prune model. One disadvantage of using human-made criteria to prune filters is that the design and selection of threshold criteria depend on complicated prior knowledge. Besides, the pruning process is less robust due to the impact of directly regularizing on filters. To address the problems mentioned, we propose an effective one-stage pruning framework: introducing a trainable collaborative layer to jointly prune and learn neural networks in one go. In our framework, we first add a binary collaborative layer for each original filter. Then, a new type of gradient estimator - asymptotic gradient estimator is first introduced to pass the gradient in the binary collaborative layer. Finally, we simultaneously learn the sparse structure and optimize the weights from the original model in the training process. Our evaluation results on typical benchmarks, CIFAR and ImageNet, demonstrate very promising results against other state-of-the-art filter pruning methods.

中文翻译：

---

通过培训进行修剪：一种用于图像处理的新型深度神经网络压缩框架

预训练卷积神经网络的过滤器修剪通常是通过人为约束或准则（例如规范，等级等）执行的。通常，修剪流程包括两个阶段：首先从原始模型中学习稀疏结构，然后在新修剪模型中优化权重。使用人工标准修剪过滤器的一个缺点是阈值标准的设计和选择取决于复杂的先验知识。此外，由于直接对过滤器进行正则化的影响，修剪过程的鲁棒性较差。为了解决上述问题，我们提出了一个有效的一阶段修剪框架：引入可训练的协作层，以一次性修剪和学习神经网络。在我们的框架中，我们首先为每个原始过滤器添加一个二进制协作层。然后，首先引入一种新型的梯度估计器-渐近梯度估计器，以在二进制协作层中传递梯度。最后，我们在训练过程中同时从原始模型中学习稀疏结构并优化权重。我们在典型基准（CIFAR和ImageNet）上的评估结果证明，与其他最先进的滤镜修剪方法相比，它们的结果很有希望。