This paper proposes a new learning paradigm called filter grafting, which aims to improve the representation capability of Deep Neural Networks (DNNs). The motivation is that DNNs have unimportant (invalid) filters (e.g., l1 norm close to 0). These filters limit the potential of DNNs since they are identified as having little effect on the network. While filter pruning removes these invalid filters for efficiency consideration, filter grafting re-activates them from an accuracy boosting perspective. The activation is processed by grafting external information (weights) into invalid filters. To better perform the grafting process, we develop an entropy-based criterion to measure the information of filters and an adaptive weighting strategy for balancing the grafted information among networks. After the grafting operation, the network has very few invalid filters compared with its untouched state, enpowering the model with more representation capacity. We also perform extensive experiments on the classification and recognition tasks to show the superiority of our method. For example, the grafted MobileNetV2 outperforms the non-grafted MobileNetV2 by about 7 percent on CIFAR-100 dataset. Code is available at https://github.com/fxmeng/filter-grafting.git.

中文翻译：

---

深度神经网络的滤波器嫁接

本文提出了一种称为滤波器嫁接的新学习范式，旨在提高深度神经网络 (DNN) 的表示能力。动机是 DNN 具有不重要（无效）的过滤器（例如，l1 范数接近于 0）。这些过滤器限制了 DNN 的潜力，因为它们被认为对网络几乎没有影响。虽然出于效率考虑，过滤器修剪会删除这些无效过滤器，但过滤器嫁接从提高准确性的角度重新激活它们。通过将外部信息（权重）嫁接到无效过滤器中来处理激活。为了更好地执行嫁接过程，我们开发了一种基于熵的标准来衡量过滤器的信息，并开发了一种自适应加权策略来平衡网络之间的嫁接信息。嫁接手术后，与未触及的状态相比，网络的无效过滤器很少，从而使模型具有更多的表示能力。我们还在分类和识别任务上进行了大量实验，以展示我们方法的优越性。例如，嫁接的 MobileNetV2 在 CIFAR-100 数据集上的性能比未嫁接的 MobileNetV2 高约 7%。代码可在 https://github.com/fxmeng/filter-grafting.git 获得。