Automatic second language (L2) speech fluency assessment has been one of the ultimate goals of several projects aiming at designing Computer-Assisted Pronunciation Training (CAPT) tools for L2 learners. Usually, three challenges must be tackled in order to solve the issues at stake: 1) Defining fluency from a threefold interdisciplinary perspective (acoustic and perceptual phonetics, computer science

Dysarthria is a motor speech disorder leading to imprecise articulation of speech. Acoustic analysis capable of detecting and assessing articulation errors is useful in dysarthria diagnosis and therapy. Since speakers with dysarthria experience difficulty in producing rhotics due to complex articulatory gestures of these sounds, the hypothesis of the present study is that duration of the rhotic approximant

We address the problem of suppressing musical noise from speech enhanced using a short-time processing algorithm. Enhancement algorithms rely on noise statistics and errors in estimating the statistics lead to residual noise in the enhanced signal. A frequently encountered residual noise type is the so-called musical noise, which is a consequence of spurious peaks occurring at random locations in the

This paper presents text tokenization and context extraction without using language knowledge for text-to-speech (TTS) synthesis. To generate prosody, statistical parametric TTS synthesis typically requires the professional knowledge of the target language. Therefore, languages suitable for TTS synthesis are limited to only rich-resource languages. To achieve TTS synthesis without using language knowledge

In dialogue, speakers produce and perceive acoustic/prosodic turn-taking cues, which are fundamental for negotiating turn exchanges with their interlocutors. However, little of the temporal dynamics and cross-linguistic validity of these cues is known. In this work, we explore a set of acoustic/prosodic cues preceding three turn-transition types (hold, switch and backchannel) in three different languages

Nonnative sound perception might be challenging for adult listeners since they attune from a very young age to the phonological aspects of their native language and, thus, every nonnative sound is filtered through their first language. The present study investigates the perception of Greek consonants in both Consonant-Vowel (CV) and Vowel-Consonant (VC) syllable context by Russian monolingual speakers

Building dialogue systems plays an important role in modern life. amongst them, task-oriented dialogue for resolving problems in actual life is most worth exploring. Motivated by the development of end-to-end approaches, a task-oriented dialogue model based on bidirectional LSTM and self-attention mechanism is proposed. It not only makes good use of context and effectively solves the long-term dependency

Behavioral studies on concurrent vowels with shorter duration (50 ms) show that overall identification scores across fundamental frequency (F0) differences are reduced when compared to longer duration (200 ms). In this current study, we investigated the effect of shorter durations on concurrent vowel scores using the temporal responses of an auditory-nerve model (Zilany et al., 2014) with a modified

The present study investigates relationships between voice similarity ratings made by human listeners and comparison scores produced by an automatic speaker recognition system that includes phonetic, perceptually-relevant features in its modelling. The study analyses human voice similarity ratings of pairs of speech samples from unrelated speakers from an accent-controlled database (DyViS, Standard

In the existing speech enhancement methods based on deep neural network (DNN), the network architectures are not designed for speech enhancement specially, which extract local features of noisy speech in a non-causal way. In this paper, inspired by the feature calculation method based on the time–frequency correlation in the improved minima controlled recursive averaging (IMCRA), by using the long

The current study describes the collection of a new phonemically-balanced sentence resource for French, known as the Fharvard corpus. The resource consists of 700 sentences inspired by the original English Harvard sentences, along with audio recordings from one female and one male native French talker. Each of the sentences contains five mono- or bisyllabic keywords and are grouped into 70 lists of

Entrainment is the tendency of interlocutors to become more similar to each other in their way of speaking. This phenomenon has been repeatedly documented and is associated with multiple social aspects of human-human conversations. However, there is a dearth of research on the effects of spoken dialogue systems (SDSs) with implemented acoustic-prosodic (dis)entrainment policies. The goal of the present

The frequency-domain adaptive filter (FDAF) is a competitive candidate for stereo acoustic echo cancellation (SAEC) because of its good convergence and computational efficiency. However, there is a conflict between the convergence rate and the steady-state misalignment when using a constant step size. In this paper, a variable step-size approach is proposed for the stereo echo canceller in the frequency

Feature representation plays a critical role in speech emotion recognition (SER). As a method of data dimensionality reduction, Non-negative Matrix Factorization (NMF) can obtain the low-dimensional representation of data by matrix decomposition, and make the data more distinguishable. In order to improve the recognition ability of NMF for SER, we conduct a potential study on NMF and propose a supervised

Hypernasality in the speech of children with cleft palate is a consequence of velopharyngeal insufficiency. The spectral analysis of hypernasal speech shows the presence of nasal formants and anti-formants in the spectrum which affects the harmonic-intensity. The nasal formants increase whereas the anti-formants decrease the magnitude of harmonics around its location of addition. Hence, the spectrum

Research on prominence perception has made use of animated agents and controlled speech in experimental settings, but these methodologies have disregarded some aspects of the acoustic and visual correlates of prominence. To overcome these limitations we propose a new methodological approach using spontaneous speech data. For this, we created a small database with extracts from a television talent show

Speech signals may contain different paralinguistic aspects such as the presence of pathologies that affect the proper communication capabilities of a speaker. Those speech disorders have different origin depending on the type of the disease. For instance, diseases with morphological origin such as cleft lip and palate that causes hypernasality, or with neurodegenerative origin such as Parkinson’s

The validity of glottal inverse filtering (GIF) to obtain a glottal flow waveform from radiated pressure signal in the presence and absence of source-filter interaction was studied systematically. A driven vocal fold surface model of vocal fold vibration was used to generate source signals. A one-dimensional wave reflection algorithm was used to solve for acoustic pressures in the vocal tract. Several

In this paper we evaluate the accuracy, recording interference, and articulatory quality of two different ultrasound probe stabilization headsets: a metallic Ultrasound Stabilisation Headset (USH) and UltraFit, a recently developed headset that is 3D printed in Nylon. To evaluate accuracy, we recorded three native speakers of German with different head sizes using an optical marker tracking system

The speech of the individuals with cleft palate (CP) is generally characterized by the presence of abnormal nasal resonances during the production of voiced sounds, primarily in vowels, and is called hypernasality. Hypernasality is present in more than 50% of the individuals with CP, and it often results in degraded speech, both in quality and intelligibility. The current work describes the signal

The aim of the present perceptual study is to weight tones and vowels as acoustic cues in Chinese subregional dialect identification, and to test the credibility of the subregional dialect classification that has been proposed in the literature. Our findings show that listeners are able to pinpoint speakers’ subregional dialect even when only given monosyllabic Chinese word stimuli, either natural

Objective intelligibility assessment of dysarthric speech can assist clinicians in diagnosis of speech disorders as well as in medical treatment. This study investigates the use of glottal parameters (i.e. parameters that describe the acoustical excitation of voiced speech, the glottal flow) in the automatic intelligibility assessment of dysarthric speech. Instead of directly predicting the intelligibility

The electrolarynx (EL) is an electromechanical device that enables patients to produce voice following the surgical removal of their larynx. The purpose of this study is to understand the acoustic and perceptual characteristics of Mandarin consonants produced by EL speakers. First, the acoustic characteristics (including speech intensity, consonant duration, spectral peak, and F2 onset) of Mandarin

In this paper, we investigate the application of graph signal processing (GSP) theory in speech enhancement. We first propose a set of shift operators to construct graph speech signals, and then analyze their spectrum in the graph Fourier domain. By leveraging the differences between the spectrum of graph speech and graph noise signals, we further propose the graph spectral subtraction (GSS) method

Building speech synthesis systems for Indian languages is challenging owing to the fact that digital resources for these languages are hardly available. Vocabulary independent speech synthesis requires that a given text is split at the level of the smallest sound unit, namely, phone. The waveforms or models of phones are concatenated to produce speech. The waveforms corresponding to that of the phones

In this study, we propose a new concept, the gammachirp envelope distortion index (GEDI), based on the signal-to-distortion ratio in the auditory envelope, SDRenv, to predict the intelligibility of speech enhanced by nonlinear algorithms. The objective of GEDI is to calculate the distortion between enhanced and clean-speech representations in the domain of a temporal envelope extracted by the gammachirp

We present a novel study of relationships between automatic accent identification (AID) and accent-robust automatic speech recognition (ASR), using i-vector based AID and deep neural network, hidden Markov Model (DNN-HMM) based ASR. A visualization of the AID i-vector space and a novel analysis of the accent content of the WSJCAM0 corpus are presented. Accents that occur at the periphery of AID space

A deep neural network approach to voice conversion usually depends on a large amount of parallel training data from source and target speakers. In this paper, we propose a novel conversion pipeline, DeepConversion, that leverages a large amount of non-parallel, multi-speaker data, but requires only a small amount of parallel training data. It is believed that we can represent the shared characteristics

This paper presents a revised version of the Hearing-Aid Speech Perception Index (HASPI). The index is based on a model of the auditory periphery that incorporates changes due to hearing loss and is valid for both normal-hearing and hearing-impaired listeners. It is an intrusive metric that compares the time-frequency envelope and temporal fine structure (TFS) of a degraded signal to an unprocessed

Speech emotion recognition (SER) has recently been receiving increased interest due to the rapid advancements in affective computing and human computer interaction. English, German, Mandarin and Indian are among the most commonly considered languages for SER along with other European and Asian languages. However, few researches have implemented Arabic SER systems due to the scarcity of available Arabic

This paper presents a review of multi-objective deep learning methods that have been introduced in the literature for speech denoising. After stating an overview of conventional, single objective deep learning, and hybrid or combined conventional and deep learning methods, a review of the mathematical framework of the multi-objective deep learning methods for speech denoising is provided. A representative

Previous studies have suggested that female voices may impede verbal processing. For example, words were remembered less well and lexical decision was slower when spoken by a female speaker. The current study tried to replicate this gender effect in an auditory semantic/associative priming task that excluded any effects of speaker variability and extended previous research by examining the role of

This study investigates the relative contribution of L2 intonation, rhythm, and speech rate to L1 perceptions of accentedness and comprehensibility. The intonation, rhythm, and speech rate of an L1 speaker of Dutch was transferred onto the segmental string of four Spanish learners of Dutch, resulting in eight conditions that reflect all possible combinations of these prosodic cues. Our results show

In this paper, we investigate single-channel speech enhancement algorithms that operate in the short-time Fourier transform and take into account dependencies w.r.t. frequency. As a result of allowing for inter-frequency dependencies, the minimum mean square error optimal estimates of the short-time Fourier transform expansion coefficients are functions of complex-valued covariance matrices in general

Spoken documents, such as podcasts or lectures, are a growing presence in everyday life. Being able to automatically identify their discourse structure is an important step to understanding what a spoken document is about. Moreover, finer-grained units, such as paragraphs, are highly desirable for presenting and analyzing spoken content. However, little work has been done on discourse based speech

This paper builds on a multi-task Deep Neural Network (DNN), which provides an utterance-level feature representation called j-vector, to implement a Text-dependent Speaker Verification (TDSV) system. This technique exploits the speaker idiosyncrasies associated with individual pass-phrases. However, speaker information is known to be characteristic of more specific speech units and, thus, it is likely

The objectives of the dysarthria assessment are to discriminate dysarthric speech from normal speech, to estimate the severity of dysarthria in terms of the dysarthric speech intelligibility, and to find the motor speech subsystem which causes defects in speech production. In this work, analytic phase features are investigated for the objective assessment of dysarthria. In this connection, the importance

Many paralinguistic applications of speech demand the extraction of information about the speaker characteristics from as little speech data as possible. In this work, we explore the estimation of multiple physical parameters of the speaker from the short duration of speech in a multilingual setting. We explore different feature streams for age and body build estimation derived from the speech spectrum

This work presents a Kronecker product based methodology of frequency-domain beamforming of large sensor arrays for far-field broadband speech signals. The principal idea involves splitting up a given uniform linear array (ULA) into two smaller virtual ULAs (VULAs), using the Kronecker product. The linear system of the original ULA is bifurcated into two smaller linear systems of the VULAs. Henceforth

Deep learning has become one of the most widely accepted paradigms regarding machine learning. It focuses on the use of hierarchical data models and builds upon the notion that in order to learn about high level data representations, a better understanding of intermediate level representation is needed. Restricted Boltzmann Machines and deep belief networks are two main types of deep learning algorithms

Speech emotion recognition plays an increasingly important role in emotional computing and is still a challenging task due to its complexity. In this study, we developed a framework integrating three distinctive classifiers: a deep neural network (DNN), a convolution neural network (CNN), and a recurrent neural network (RNN). The framework was used for categorical recognition of four discrete emotions

In linear predictive speech coders, the linear predictive coding (LPC) parameters are usually transformed into the line spectral frequency (LSF) representation for quantization. In this paper, the single and multiple frame coding of LSF parameters using deep neural network (DNN) and pyramid vector quanrizer (PVQ) are proposed. In the single-frame scheme, a non-linear DNN predictor which demonstrates

We propose an information theoretic framework for quantitative assessment of acoustic models used in hidden Markov model (HMM) based automatic speech recognition (ASR). The HMM backend expects that (i) the acoustic model yields accurate state conditional emission probabilities for the observations at each time step, and (ii) the conditional probability distribution of the data given the underlying

In this study we investigated Convolutional Neural Networks (CNNs) for the classification of infant vocalization sequences. The target classes were ‘crying’, ‘fussing’, ‘babbling’, ‘laughing’ and ‘vegetative vocalizations’. The general case of this classification task is of importance for applications which require a qualitative evaluation of general infant vocalizations, such as pain assessment or

This paper presents a study of continuous word representations applied to automatic detection of speech recognition errors. A neural network architecture is proposed, which is well suited to handle continuous word representations, like word embeddings. We explore the use of several types of word representations: simple and combined linguistic embeddings, and acoustic ones associated to prosodic features

The aim of this paper is to develop a flexible framework capable of automatically recognizing phonetic units present in a speech utterance of any language spoken in any mode. In this study, we considered two modes of speech: conversation and read modes in four Indian languages, namely, Telugu, Kannada, Odia, and Bengali. The proposed approach consists of two stages: (i) Automatic speech mode classification

In human-to-human communication, speech signals carry rich emotional cues that are further emphasized by affect-expressive gestures. In this regard, automatic synthesis and animation of gestures accompanying affective verbal communication can help to create more naturalistic virtual agents in human-computer interaction systems. Speech-driven gesture synthesis can map emotional cues of the speech signal

Both in speech and singing, humans are capable of generating sounds of different phonation types (e.g., breathy, modal and pressed). Previous studies in the analysis and classification of phonation types have mainly used voice source features derived using glottal inverse filtering (GIF). Even though glottal source features are useful in discriminating phonation types in speech, their performance deteriorates

The task of forensic voice comparison (FVC) often involves the comparison of a voice in an offender recording with that in a suspect recording, with the aim to assist the investigating authority or the court in determining the identity of the speaker. One of the main goals in FVC research is to identify speech variables that are useful for differentiating speakers. While French and Stevens (2013) stated

The performance of speaker verification depends on the overlap region of the decision scores of true and imposter trials. Motivated by the fact that the overlap region can be reduced by maximizing the between-class distance while minimizing the within-class variance of the trials, we present in this paper two cosine metric learning (CML) back-end algorithms. The first one, named m-CML, aims to enlarge

Mandarin wh-words can have question or non-question (e.g., existential, universal quantificational) interpretations. Their interpretations in a sentence are usually not ambiguous, as the distinct interpretations need to be licensed by particular items/contexts. The starting point of our study concerns a case which allows the wh-words to remain ambiguous in a sentence: wh-words such as shénme appearing

Speech enhancement aims to improve the quality and intelligibility of speech signals, which is a challenging task in adverse environments. Speech enhancement generative adversarial network (SEGAN) that adopted a generative adversarial network (GAN) for speech enhancement achieved promising results. In this paper, a new network architecture and loss function based on SEGAN are proposed for speech enhancement

This paper focuses on adaptable continuous space language modeling approach of combining longer context information of recurrent neural network (RNN) with adaptation ability of subspace Gaussian mixture model (SGMM) which has been widely used in acoustic modeling for automatic speech recognition (ASR). In large vocabulary continuous speech recognition (LVCSR) it is a challenging problem to construct

In noisy conditions, speakers involuntarily change their manner of speaking to enhance the intelligibility of their voices. The increased intelligibility of this so-called Lombard speech is enabled by the change of multiple articulatory and acoustic features. While the major features of Lombard speech are well known from previous studies, little is known about their relative contributions to the intelligibility

The second language learning process involves acquisition of sounds that differ to varying degrees from the sounds of a learner’s native language. Learners’ productions are strongly influenced by their native language particularly for sounds which are similar but non-identical in the two languages. However, foreign accent is typically investigated at the level of utterances and as a consequence the

This paper focuses on the process whereby speakers position themselves in jointly produced conversations. The expected degree of dominancy (degree of power realization) in the dialogues is derived by the independent variable of the role of a participant – a leader or a follower – in a Map Task setting. We examine the participants’ dominancy as reflected by a set of structural and lexical features.

Speech-based computer-assisted language learning (CALL) systems should recognize the utterances of the learner with high accuracy and evaluate the language proficiency of the specific speaker with appropriate methods. In this paper, we discuss the automatic assessment of the second language (L2) for non-native speakers. There are many existing works on pronunciation evaluation by applying the goodness

Speech is the most natural way of expressing ourselves as humans. It is only natural then to extend this communication medium to computer applications. We define speech emotion recognition (SER) systems as a collection of methodologies that process and classify speech signals to detect the embedded emotions. SER is not a new field, it has been around for over two decades, and has regained attention

Fundamental frequency (f0) extraction plays an important role in processing of monophonic signals such as speech and song. It is essential in various real-time applications such as emotion recognition, speech/singing voice discrimination and so on. Several f0 extraction methods have been proposed over the years, but no one algorithm works well for both speech and song. In this paper, we propose a novel

This paper presents a study where a 3D motion-capture animated ‘virtual speaker’ is compared to a video of a real speaker with regards to how it facilitates children's speech comprehension of narratives in background multitalker babble noise. As secondary measures, children self-assess the listening- and attentional effort demanded by the task, and associates words describing positive or negative social