In this paper, we present a streaming end-to-end speech recognition model based on Monotonic Chunkwise Attention (MoCha) jointly trained with enhancement layers. Even though the MoCha attention enables streaming speech recognition with recognition accuracy comparable to a full attention-based approach, training this model is sensitive to various factors such as the difficulty of training examples, hyper-parameters, and so on. Because of these issues, speech recognition accuracy of a MoCha-based model for clean speech drops significantly when a multi-style training approach is applied. Inspired by Curriculum Learning [1], we introduce two training strategies: Gradual Application of Enhanced Features (GAEF) and Gradual Reduction of Enhanced Loss (GREL). With GAEF, the model is initially trained using clean features. Subsequently, the portion of outputs from the enhancement layers gradually increases. With GREL, the portion of the Mean Squared Error (MSE) loss for the enhanced output gradually reduces as training proceeds. In experimental results on the LibriSpeech corpus and noisy far-field test sets, the proposed model with GAEF-GREL training strategies shows significantly better results than the conventional multi-style training approach.

中文翻译：

---

具有联合训练的神经特征增强功能的流式端到端语音识别

在本文中，我们提出了一种基于增强层联合训练的单调块状注意（MoCha）的流式端到端语音识别模型。尽管MoCha注意可以实现流语音识别，其识别准确度可与基于完全注意的方法相媲美，但是训练此模型对各种因素敏感，例如训练示例的难度，超参数等。由于这些问题，采用多样式训练方法时，基于MoCha的干净语音模型的语音识别准确性会大大降低。受课程学习[1]的启发，我们引入了两种培训策略：逐步应用增强功能（GAEF）和逐步减少增强损失（GREL）。借助GAEF，该模型最初是使用干净特征进行训练的。随后，增强层的输出部分逐渐增加。使用GREL，随着训练的进行，增强输出的均方误差（MSE）损失部分逐渐减少。在LibriSpeech语料库和嘈杂的远场测试集的实验结果中，提出的具有GAEF-GREL训练策略的模型显示出比传统的多样式训练方法明显更好的结果。