Unseen or out-of-domain data can seriously degrade the performance of a neural network model, indicating the model's failure to generalize to unseen data. Neural net pruning can not only help to reduce a model's size but can improve the model's generalization capacity as well. Pruning approaches look for low-salient neurons that are less contributive to a model's decision and hence can be removed from the model. This work investigates if pruning approaches are successful in detecting neurons that are either high-salient (mostly active or hyper) or low-salient (barely active or hypo), and whether removal of such neurons can help to improve the model's generalization capacity. Traditional blind adaptation techniques update either the whole or a subset of layers, but have never explored selectively updating individual neurons across one or more layers. Focusing on the fully connected layers of a convolutional neural network (CNN), this work shows that it may be possible to selectively adapt certain neurons (consisting of the hyper and the hypo neurons) first, followed by a full-network fine tuning. Using the task of automatic speech recognition, this work demonstrates how the removal of hyper and hypo neurons from a model can improve the model's performance on out-of-domain speech data and how selective neuron adaptation can ensure improved performance when compared to traditional blind model adaptation.

中文翻译：

---

Hyper和Hypo神经元修剪在无监督适应过程中选择性更新神经元的调查与分析

不可见或域外数据会严重降低神经网络模型的性能，表明该模型无法泛化到不可见数据。神经网络修剪不仅可以帮助减小模型的大小，还可以提高模型的泛化能力。修剪方法寻找对模型决策贡献较小的低显着神经元，因此可以从模型中删除。这项工作调查了修剪方法是否能够成功检测高显着性（主要是活跃或高度活跃）或低显着性（几乎不活跃或低）的神经元，以及去除这些神经元是否有助于提高模型的泛化能力。传统的盲适应技术更新整个层或部分层，但从未探索过跨一层或多层选择性地更新单个神经元。专注于卷积神经网络 (CNN) 的全连接层，这项工作表明有可能首先选择性地适应某些神经元（由超和超神经元组成），然后是全网络微调。使用自动语音识别任务，这项工作展示了从模型中去除超神经元和低神经元如何提高模型在域外语音数据上的性能，以及与传统的盲模型相比，选择性神经元自适应如何确保提高性能适应。这项工作表明，有可能首先选择性地适应某些神经元（由超神经元和低神经元组成），然后是全网络微调。使用自动语音识别任务，这项工作展示了从模型中去除超神经元和低神经元如何提高模型在域外语音数据上的性能，以及与传统的盲模型相比，选择性神经元自适应如何确保提高性能适应。这项工作表明，有可能首先选择性地适应某些神经元（由超神经元和低神经元组成），然后是全网络微调。使用自动语音识别任务，这项工作展示了从模型中去除超神经元和低神经元如何提高模型在域外语音数据上的性能，以及与传统的盲模型相比，选择性神经元自适应如何确保提高性能适应。