Time–frequency representations of the speech signals provide dynamic information about how the frequency component changes with time. In order to process this information, deep learning models with convolution layers can be used to obtain feature maps. In many speech processing applications, the time–frequency representations are obtained by applying the short-time Fourier transform and using single-channel input tensors to feed the models. However, this may limit the potential of convolutional networks to learn different representations of the audio signal. In this paper, we propose a methodology to combine three different time–frequency representations of the signals by computing continuous wavelet transform, Mel-spectrograms, and Gammatone spectrograms and combining then into 3D-channel spectrograms to analyze speech in two different applications: (1) automatic detection of speech deficits in cochlear implant users and (2) phoneme class recognition to extract phone-attribute features. For this, two different deep learning-based models are considered: convolutional neural networks and recurrent neural networks with convolution layers.

语音信号的时频表示提供了有关频率分量如何随时间变化的动态信息。为了处理此信息，可以使用具有卷积层的深度学习模型来获取特征图。在许多语音处理应用程序中，通过应用短时傅立叶变换并使用单通道输入张量来馈送模型，可以获得时频表示。但是，这可能会限制卷积网络学习音频信号的不同表示形式的潜力。在本文中，我们提出了一种通过计算连续小波变换，梅尔谱图，和Gammatone频谱图，然后组合成3D通道频谱图，以分析两种不同应用程序中的语音：（1）自动检测人工耳蜗用户的语音缺陷，以及（2）音素类别识别以提取电话属性。为此，考虑了两种不同的基于深度学习的模型：卷积神经网络和具有卷积层的递归神经网络。