This work addresses the problem of network pruning and proposes a novel joint training method based on a multiobjective optimization model. Most of the state-of-the-art pruning methods rely on user experience for selecting the sparsity ratio of the weight matrices or tensors, and thus suffer from severe performance reduction with inappropriate user-defined parameters. Moreover, networks might be inferior due to the inefficient connecting architecture search, especially when it is highly sparse. It is revealed in this work that the network model might maintain sparse characteristic in the early stage of the backpropagation (BP) training process, and evolutionary computation-based algorithms can accurately discover the connecting architecture with satisfying network performance. In particular, we establish a multiobjective sparse model for network pruning and propose an efficient approach that combines BP training and two modified multiobjective evolutionary algorithms (MOEAs). The BP algorithm converges quickly, and the two MOEAs can search for the optimal sparse structure and refine the weights, respectively. Experiments are also included to prove the benefits of the proposed algorithm. We show that the proposed method can obtain a desired Pareto front (PF), leading to a better pruning result comparing to the state-of-the-art methods, especially when the network structure is highly sparse.

这项工作解决了网络剪枝问题，并提出了一种基于多目标优化模型的新型联合训练方法。大多数最先进的剪枝方法依赖于用户经验来选择权重矩阵或张量的稀疏比，因此由于用户定义的参数不合适而导致性能严重下降。此外，由于连接架构搜索效率低下，网络可能会很差，特别是当它非常稀疏时。该工作揭示了网络模型在反向传播（BP）训练过程的早期可能保持稀疏特征，基于进化计算的算法可以准确地发现具有令人满意的网络性能的连接架构。特别是，我们建立了一个用于网络剪枝的多目标稀疏模型，并提出了一种将 BP 训练和两个修改后的多目标进化算法 (MOEA) 相结合的有效方法。BP算法收敛速度快，两个MOEA可以分别搜索最优稀疏结构和细化权重。还包括实验以证明所提出算法的好处。我们表明，与最先进的方法相比，所提出的方法可以获得理想的帕累托前沿（PF），从而获得更好的修剪结果，尤其是在网络结构高度稀疏的情况下。两个 MOEA 可以分别搜索最优稀疏结构和细化权重。还包括实验以证明所提出算法的好处。我们表明，与最先进的方法相比，所提出的方法可以获得理想的帕累托前沿（PF），从而获得更好的修剪结果，尤其是在网络结构高度稀疏的情况下。两个 MOEA 可以分别搜索最优稀疏结构和细化权重。还包括实验以证明所提出算法的好处。我们表明，与最先进的方法相比，所提出的方法可以获得理想的帕累托前沿（PF），从而获得更好的修剪结果，尤其是在网络结构高度稀疏的情况下。