Weight normalization (WN) can help to stabilize the distribution of activations over layers, which boost the performance of DNNs in generalization. In this paper, we further propose deep structural weight normalization (DSWN) methods to inject the network structure measurements into the WN to fully acknowledge the data propagation through the neural network. In DSWN, two novel structural measurements are developed to impose regularity on each network weight using different penalty matrices. One is sparsity measurement (DSWN-SM). In this measurement, L1,2 weight regularization is applied in our proposed model to promote competition for features between network weights to obtain a sparsity network and finally prune the network. The other is neuron measurement (DSWN-NM). It uses L2 norm of column weight to scale up or down the importance of each intermediate neuron, which leads to accelerating the speed of network convergence. Extensive experiments on several benchmark image datasets using fully connected network and convolution neural network are performed, and the proposed DSWN-SM and DSWN-NM methods are compared with state-of-the-art sparsity and weight normalization methods. The results show that DSWN-SM can reduce the number of trainable parameters while guaranteeing high accuracy, whereas DSWN-NM can accelerate the convergence while improving the performance of deep networks.

权重归一化（WN）可以帮助稳定激活在层上的分布，从而提高DNN的泛化性能。在本文中，我们进一步提出了深度结构权重归一化（DSWN）方法，将网络结构度量值注入WN中，以充分确认数据通过神经网络的传播。在DSWN中，开发了两种新颖的结构度量，以使用不同的罚矩阵对每个网络权重施加规则性。一种是稀疏度测量（DSWN-SM）。在这种测量中，在我们提出的模型中应用了L1,2权重正则化，以促进网络权重之间的特征竞争，从而获得稀疏网络并最终修剪网络。另一个是神经元测量（DSWN-NM）。它使用列权重的L2范数来按比例放大或缩小每个中间神经元的重要性，从而加快网络收敛的速度。使用完全连接的网络和卷积神经网络对几个基准图像数据集进行了广泛的实验，并将所提出的DSWN-SM和DSWN-NM方法与最新的稀疏性和权重归一化方法进行了比较。结果表明，DSWN-SM可以在保证高精度的同时减少可训练参数的数量，而DSWN-NM可以在提高深度网络性能的同时加快收敛速度​​。并将提出的DSWN-SM和DSWN-NM方法与最新的稀疏性和权重归一化方法进行比较。结果表明，DSWN-SM可以在保证高精度的同时减少可训练参数的数量，而DSWN-NM可以在提高深度网络性能的同时加快收敛速度​​。并将提出的DSWN-SM和DSWN-NM方法与最新的稀疏性和权重归一化方法进行比较。结果表明，DSWN-SM可以在保证高精度的同时减少可训练参数的数量，而DSWN-NM可以在提高深度网络性能的同时加快收敛速度​​。