All Italian consonants affected by gemination, that is affricates, fricatives, liquids, nasals, and stops, were analyzed within a project named GEMMA that lasted over a span of about 25 years. Results of the analysis on stops, as published in (Esposito, A., and Di Benedetto, M. G. (1999). “Acoustic and Perceptual Study of Gemination in Italian Stops,” The Journal of the Acoustical Society of America

We propose a method of automatically selecting appropriate responses in conversational spoken dialog systems by explicitly determining the correct response type that is needed first, based on a comparison of the user’s input utterance with many other utterances. Response utterances are then generated based on this response type designation (back channel, changing the topic, expanding the topic, etc

We present a database of parallel recordings of speech and singing, collected and released by the Human Language Technology (HLT) laboratory at the National University of Singapore (NUS), that is called NUS-HLT Speak–Sing (NHSS) database. We release this database1 to the public to support research activities, that include, but not limited to comparative studies of acoustic attributes of speech and

Consonant gemination in Italian affricates and fricatives was investigated, completing the overall study of gemination of Italian consonants. Results of the analysis of other consonant categories, i.e. stops, nasals, and liquids, showed that closure duration for stops and consonant duration for nasals and liquids, form the most salient acoustic cues to gemination. Frequency and energy domain parameters

Phonetic convergence involves a talker subtly sharing their speaking style with another talker. It is known that speaking style is something perceivers can learn in order to better recognize a talker and a talker's speech. The question arises, can perceivers also learn to better recognize when a talker's speaking style is being shared with another speaker during convegence? To test this question, two

This paper proposes deep Gaussian process (DGP)-based frameworks for multi-speaker speech synthesis and speaker representation learning. A DGP has a deep architecture of Bayesian kernel regression, and it has been reported that DGP-based single speaker speech synthesis outperforms deep neural network (DNN)-based ones in the framework of statistical parametric speech synthesis. By extending this method

The cochlea is a remarkable spectrum analyser with desirable properties including sharp frequency tuning and level-dependent compression and the potential advantages of incorporating these characteristics in a speech processing front-end are investigated. This paper develops a framework for an active transmission line cochlear model employing adaptive notch and resonant filters. The proposed model

Performance of Automatic Speech Recognition (ASR) systems is known to suffer considerable degradation when exposed to Far-Field speech data capture. Consequently, far-field ASR has received considerable attention in recent years. Motivated by our recent work using Curriculum Learning (CL) based strategies to improve Speaker Identification (SID) under noisy and degraded conditions, this study proposes

Electrical hearing by cochlear implants (CIs) may be fundamentally different from the acoustic hearing by normal-hearing (NH) listeners, presumably showing unequal speech quality perception in various noise environments. Noise reduction (NR) algorithms used in CIs reduce the noise in favor of signal-to-noise ratio (SNR), regardless of plausible accompanying distortions that may degrade the speech quality

Acoustic models of the vocal tract for articulatory speech synthesis often neglect a range of acoustic effects that are known to exist in the human vocal tract. Here we extended a basic acoustic vocal tract model by three features: the piriform fossae, transvelar acoustic coupling of the oral and nasal cavities, and sound radiation from the skin of the neck. The main goal was to find out how these

Ultrasound tongue imaging is used to visualise the intra-oral articulators during speech production. It is utilised in a range of applications, including speech and language therapy and phonetics research. Ultrasound and speech audio are recorded simultaneously, and in order to correctly use this data, the two modalities should be correctly synchronised. Synchronisation is achieved using specialised

The role of prosody in the encoding of information structure in Egyptian Arabic is not yet fully understood. A recent production study found significant acoustic differences between morpho-syntactically identical utterances expressing a topic-comment and a focus-background structure. This paper presents the results from two perception experiments testing Egyptian Arabic listeners’ ability to infer

Emotional dependencies play a crucial role in understanding complexities of dyadic interactions. Recent studies have shown that affect recognition tasks can benefit by the incorporation of a particular interaction’s context, however, the investigation of affect context in dyadic settings using neural network frameworks remains a complex and open problem. In this paper, we formulate the concept of dyadic

Studies that quantify speech tempo on acoustic grounds typically use one of various rate measures. The availability of multiple measurement techniques yields ‘researcher degrees of freedom’ which call the robustness of generalisations across studies into question. However, explicit assessments of the possible impact of researchers’ choices amongst the available measures are rare. In this study we attempt

The performance of the far-field speaker identification (SI) system is usually reduced by the well-known mismatch problem imposed by environmental conditions. Speech enhancement methods are known as convenient ways of resolving the mismatches created by additive noise and reverberations. Human auditory capability for identifying and segregating sounds of speakers in complex environmental conditions

Individuals that have incurred trauma due to a suicide attempt often acquire residual health complications, such as cognitive, mood, and speech-language disorders. Due to limited access to suicidal speech audio corpora, behavioral differences in patients with a history of suicidal ideation and/or behavior have not been thoroughly examined using subjective voice quality and manual disfluency measures

Intra-speaker variability, caused by emotional speech, is a real threat to the performance of speaker recognition systems. In fact, as human beings, we are constantly changing our emotional state. While many efforts have been made to increase automatic speaker verification (ASV) robustness towards channel effects or spoofing attacks, only a handful of studies have addressed the detrimental consequences

To further improve the performance of speech enhancement methods based on deep neural networks, this paper proposes a speech enhancement network that can make full use of noisy speech characteristics in the time–frequency domain. First, based on the local correlation of noisy speech in the time–frequency domain and the spatial structure of frequency features, by using a recurrent neural network (RNN)

Little is known about the cognitive capacities underlying real-time accommodation in spoken language and how they may allow conversing speakers to adapt their speech production behaviors. This study first presents a simple attunement model that incorporates hypothesized capacities, with a focus on individual variability as one of those capacities. The model makes explicit predictions about observable

Purpose We explore the validity and reliability of an Audience Response Systems (ARS)-based measure of acceptability, applied to speech produced by children with speech sound disorder (SSD). We further explore how the suggested measure relates to an ARS-based measure of intelligibility. Finally, we explore potential differences between speech-language pathologists (SLPs), untrained adults, and children

We tested the effect of co-presence on entrainment to speech rhythm, examining differences in speech rhythm convergence during a reading task, in conditions where the reading partner was present or absent. Speech rhythm was operationalized as a two-level phenomenon. At a lower level, rhythm was operationalized as regularity in the distribution of salient acoustic events (vowel onsets) and regularity

The existing loudness compensation algorithms in digital hearing aids destroy the formant structure of the speech signal easily and do not consider the residue noise when implementing loudness compensation after speech enhancement. As a result, the output speech signal fails to meet the requirements of hearing-impaired(HI) people. To solve these problems, a novel multi-channel adaptive loudness compensation

Automatic pronunciation error detection (APED) plays an important role in the domain of language learning. As for the previous ASR-based APED methods, the decoded results need to be aligned with the target text so that the errors can be found out. However, since the decoding process and the alignment process are independent, the prior knowledge about the target text is not fully utilized. In this paper

WaveNet is introduced for waveform generation. It produces high quality text-to-speech synthesis, music generation, and voice conversion. However, it generally requires a large amount of training data, that limits its scope of applications, e.g. in voice conversion. In this paper, we propose a factorized WaveNet for limited data tasks. Specifically, we apply singular value decomposition (SVD) on the

Objective quality assessment aims towards evaluating the perceptual quality of a signal using a machine-based algorithm. Due to different challenges involved in the subjective evaluation of speech quality, it is necessary to develop objective measures. The goal of any non-intrusive quality assessment metric for noise-suppressed speech is to assess the quality of a noise-suppressed signal in the absence

Objective Human interactions are related to speech and facial characteristics. It was suggested that speech signals and/or images of facial expressions may reveal human emotions and that both interact for the verification of a person's identity. The present study proposes and evaluates an automated integrated speech signal and facial image analysis system for the identification of seven different human

In recent decades, the interest in speech sciences for casual conversations has increased, and researchers have begun to investigate the characteristics of spontaneous and casual speech in different languages, such as German, Dutch, English, French, Spanish and Czech. These types of investigations are based on novel spontaneous speech corpora that allow the systematic research on large amounts of data

Previous studies have suggested that in emotionally-oriented communication such as counseling and casual conversation, non-verbal communication characteristics such as turn-taking and vocal pitch synchrony were related to formation of rapport. However, in cognitive goal-oriented communication such as group problem-solving, there are few studies investigating whether the features of non-verbal communication

It is vital to authenticate the content of speech signals to prevent the framing of innocent individuals. Furthermore, lack of speech content authentication may lead to repudiation. The research problem of authenticity vs falsehood is timely considering the recent worldwide interest in fake news and viral manipulated media. Liu & Wang proposed an algorithm to detect tamperings in speech content, based

Throat microphones (TM) are a type of skin-attached non-acoustic sensors, which are robust to environmental noise but carry a lower signal bandwidth characterization than the traditional close-talk microphones (CM). Attaining high-performance phoneme recognition is a challenging task when the training data from a degrading channel, such as TM, is limited. In this paper, we address this challenge for

Speech nasalization is achieved primarily through the opening and closing of the velopharyngeal port. However, the resultant acoustic features can also be influenced by tongue configuration. Although vowel nasalization is not contrastive in English, two previous studies have found possible differences in the oral articulation of nasal and oral vowel productions, albeit with inconsistent results. In

Sarcasm, a form of nonliteral language and a subtype of verbal irony, refers to the use of words that mean the opposite of what is intended and express negative attitudes toward persons or events. Sarcasm is prevalent in everyday conversation and the appropriate understanding of the speaker's intent is crucial for successful communication. This study investigated voice qualities associated with sarcastic

Speaker recognition continues to grow as a research challenge in the field with expanded application in commercial, forensic, educational and general speech technology interfaces. However, challenges remain, especially for naturalistic audio streams including recordings with mismatch between train and test data (i.e., when train or system development data and enrollment/test data or application data

Speech sound disorders are a common communication impairment in childhood. Because speech disorders can negatively affect the lives and the development of children, clinical intervention is often recommended. To help with diagnosis and treatment, clinicians use instrumented methods such as spectrograms or ultrasound tongue imaging to analyse speech articulations. Analysis with these methods can be

Speaking and singing are mechanisms of vocal production that have distinct articulatory properties and consequently produce sounds that are normally perceived as different. Several papers indicate that the tonal stability characteristic in singing associated with pre-defined fundamental frequency (f0) target tones, i.e., musical notes, is an important differentiating factor relatively to the observed

The desired speech detector plays an important role for controlling the speech distortion in spatial filtering based speech enhancement algorithms. However, the conventional complex coherence(CC) based algorithms can only distinguish the coherent speech and diffuse noise. To improve the performance on the scenarios that both the coherent interference and diffuse noise are present, we propose a directional

In the paper, we investigate a graph representation of speech signals and graph speech enhancement technology. Specifically, we first propose a new graph k-shift operator Ck to map speech signals into the graph domain and construct a novel graph Fourier basis by using its singular eigenvectors for speech graph signals (SGSs). On this basis, we propose an improved graph Wiener filtering method based

Recently, spontaneous speech emotion recognition has become an active and challenging research subject. This paper proposes a new method of spontaneous speech emotion recognition by using deep multimodal audio feature learning based on multiple deep convolutional neural networks (multi-CNNs). The proposed method initially generates three different audio inputs for multi-CNNs so as to learn deep multimodal

Speech perception and spoken word recognition are not only affected by what is being said, but also by who is speaking. Currently, publicly available corpora of spoken Dutch do not offer a wide variety of linguistic materials produced by multiple talkers. The VariaNTS (Variatie in Nederlandse Taal en Sprekers) corpus is a Dutch spoken corpus that was developed to maximize both linguistic and talker

The present study investigates whether native speakers of German phonetically accommodate to natural and synthetic voices in a shadowing experiment. We aim to determine whether this phenomenon, which is frequently found in HHI, also occurs in HCI involving synthetic speech. The examined features pertain to different phonetic domains: allophonic variation, schwa epenthesis, realization of pitch accents

The performance of voice-based Parkinson’s disease (PD) detection systems degrades when there is an acoustic mismatch between training and operating conditions caused mainly by degradation in test signals. In this paper, we address this mismatch by considering three types of degradation commonly encountered in remote voice analysis, namely background noise, reverberation and nonlinear distortion, and

In this paper, a multilingual automatic speech recognition (ASR) and language identification (LID) system is designed. In contrast to conventional multilingual ASR systems, this paper takes advantage of the complementarity of the ASR and LID modules. First, the LID module contributes to the language adaptive training of the multilingual acoustic model. Then, the ASR decoding information acts as the

Burushaski is an endangered language isolate spoken in Hunza, Nager, and Yasin valleys of Gilgit, Northern Pakistan. The present study investigates the acoustic correlates of Hunza Burushaski’s three-way stop laryngeal contrast (voiceless unaspirated, voiceless aspirated, and voiced unaspirated) across five places of articulation (bilabial, dental, retroflex, palatal, and velar). A wide range of acoustic

There is ample evidence that both native and non-native listeners deal with speech variation by quickly tuning into a speaker and adjusting their phonetic categories according to the speaker’s ambiguous pronunciation. This process is called lexically-guided perceptual learning. Moreover, the presence of noise in the speech signal has previously been shown to change the word competition process by increasing

Communicating emotion is essential in building and maintaining relationships. We communicate our emotional state not just with the words we use, but also how we say them. Changes in the rate of speech, short-term energy and intonation all help to convey emotional states like 'angry', 'sad' and 'happy'. People with dysarthria, the most common speech disorder, have reduced articulatory and phonatory

This paper proposes architectures that facilitate the extrapolation of emotional expressions in deep neural network (DNN)-based text-to-speech (TTS). In this study, the meaning of “extrapolate emotional expressions” is to borrow emotional expressions from others, and the collection of emotional speech uttered by target speakers is unnecessary. Although a DNN has potential power to construct DNN-based

The improvement of the performance of online separating speech and music is an NP problem and the separation optimization increases the complexity of the method in a Robust Principal Component Analysis (RPCA) method which is time consuming in big size matrix computations. This paper presents a RPCA-based speech and music separation method to reduce the amount of computational complexity and be robust

Multimodal emotion recognition is a challenging task due to different modalities emotions expressed during a specific time in video clips. Considering the existed spatial-temporal correlation in the video, we propose an audio-visual fusion model of deep learning features with a Mixture of Brain Emotional Learning (MoBEL) model inspired by the brain limbic system. The proposed model is composed of two

Automatic speech emotion recognition (SER) by a computer is a critical component for more natural human-machine interaction. As in human-human interaction, the capability to perceive emotion correctly is essential to taking further steps in a particular situation. One issue in SER is whether it is necessary to combine acoustic features with other data such as facial expressions, text, and motion capture

The purpose of the study was to examine effects of verbalization of affective experience on heart-rate variability and emotions (HRV). We measured HRV of 35 subjects while they were viewing and verbalizing 48 affective pictures (IAPS). Our results showed that sympathetic activity was lower and parasympathetic activity higher when subjects were verbalizing unpleasant pictures than it was when they were

In this paper, we propose a new speech enhancement system using a deep neural network (DNN)-augmented colored-noise Kalman filter. In our system, both clean speech and noise are modelled as autoregressive (AR) processes, whose parameters comprise the linear prediction coefficients (LPCs) and the driving noise variances. The LPCs are obtained through training a multi-objective DNN that learns the mapping

Prosodic events such as pitch accents and phrase boundaries have various acoustic and temporal correlates that are used as features in machine learning models to automatically detect these events from speech. These features are often linguistically motivated, high-level features that are hand-crafted by experts to best represent the prosodic events to be detected or classified. An alternative approach

Accurately modelling the long-term dependencies of noisy speech is critical to the performance of a speech enhancement system. Current deep learning approaches to speech enhancement employ either a recurrent neural network (RNN) or a temporal convolutional network (TCN). However, RNNs and TCNs both demonstrate deficiencies when modelling long-term dependencies. Enter multi-head attention (MHA) — a

This paper presents Voks, a new family of digital instruments that allow for real-time control and modification of pre-recorded voice signal samples. An instrument based on Voks is made of Voks itself, the synthesis software and a given set of chironomic (hand-driven) interfaces. Rhythm can be accurately controlled thanks to a new methodology, based on syllabic control points. Timing can also be controlled

Replay attack poses a great threat to the Automatic Speaker Verification (ASV) system. This paper introduces Amplitude Modulation and Frequency Modulation-based features for replay Spoof Speech Detection (SSD) task. In this context, we propose Instantaneous Amplitude (IA) and Instantaneous Frequency (IF) features using Energy Separation Algorithm (ESA). The speech signal is passed through bandpass

This study examines the acoustic outputs in boundary coarticulation and attempts to explain the native and nonnative production differences in relation to speech planning. English consonant clusters across words produced by English (L1) and Mandarin (L2) speakers are analyzed. This study focuses on two segmental features (place of articulation and voicing) and computes three acoustic-based measurements

Automatic second language (L2) speech fluency assessment has been one of the ultimate goals of several projects aiming at designing Computer-Assisted Pronunciation Training (CAPT) tools for L2 learners. Usually, three challenges must be tackled in order to solve the issues at stake: 1) Defining fluency from a threefold interdisciplinary perspective (acoustic and perceptual phonetics, computer science

Dysarthria is a motor speech disorder leading to imprecise articulation of speech. Acoustic analysis capable of detecting and assessing articulation errors is useful in dysarthria diagnosis and therapy. Since speakers with dysarthria experience difficulty in producing rhotics due to complex articulatory gestures of these sounds, the hypothesis of the present study is that duration of the rhotic approximant

We address the problem of suppressing musical noise from speech enhanced using a short-time processing algorithm. Enhancement algorithms rely on noise statistics and errors in estimating the statistics lead to residual noise in the enhanced signal. A frequently encountered residual noise type is the so-called musical noise, which is a consequence of spurious peaks occurring at random locations in the