Special Issue “Understanding Others”: Research ReportRecognition of emotion from subtle and non-stereotypical dynamic facial expressions in Huntington's disease
Introduction
Huntington's Disease (HD) is an inherited neurodegenerative disorder, caused by an expanded CAG trinucleotide repetition on chromosome 4, and is characterized by progressive motor dysfunction, cognitive decline, and affective disturbance (Walker, 2007). HD is associated with neuronal loss within corticostriatal circuits (Lawrence, Sahakian, & Robbins, 1998), as neurodegeneration originates in the striatum, a component of the basal ganglia (Aylward et al., 2000, Paulsen et al., 2010). Dysfunction and death of neurons in other brain regions, including the cerebral cortex, subcortical white matter, amygdala, insula, thalamus and hypothalamus, are also evident, though atrophy in these regions is less severe than in the striatum (Dogan et al., 2013, Dumas et al., 2012, Ross and Tabrizi, 2011).
Patients are diagnosed with HD based on the presence of unequivocal motor signs, particularly chorea, in conjunction with a positive genetic testing (McColgan & Tabrizi, 2018). This diagnosis, based on the manifestation of motor symptoms, represents the onset of the disease. The manifest period is preceded by a pre-manifest period, which can be further subdivided into two clinical stages: at the first stage, pre-symptomatic HD gene carriers are clinically similar to typical controls (Ross et al., 2014). However, around 10–20 years before disease onset, subtle cognitive and behavioral alterations are frequently demonstrated (Paulsen et al., 2008, Scahill et al., 2013). This second stage of the pre-manifest period is called the prodromal stage of HD (Ross & Tabrizi, 2011).
Amongst the cognitive-affective symptoms of HD, patients sometimes experience deficits in social cognition and emotion processing (Bora, Velakoulis, & Walterfang, 2016). These deficits include social-neuropsychiatric symptoms such as apathy and disinhibition (Kempnich et al., 2018), alterations in the way emotions are experienced (Ille, Holl, Kapfhammer, Reisinger, Schäfer & Schienle, 2011), emotional blunting (Craufurd, Thompson, & Snowden, 2001), poor theory of mind (Eddy & Rickards, 2015), impaired emotion regulation (Zarotti, Fletcher, & Simpson, 2018), and a tendency to draw faulty inferences from social situations (Snowden et al., 2003). As a result, individuals with HD commonly experience reduced social quality of life, including social isolation, social withdrawal, disrupted relationships and disturbances in social interactions (Kempnich et al., 2018).
Specifically, HD patients are impaired in their ability to recognize negative facial expressions (Bora et al., 2016). This well-established reduction in negative emotion recognition in symptomatic HD patients is already detectable at the prodromal stage of the disease, in which performance is slightly impaired (Bora et al., 2016, Johnson et al., 2007, Stout et al., 2011). Early at the prodromal stage of the disease this finding is less consistently demonstrated, and is sometimes limited to some of the negative emotions (Henley et al., 2012, Zarotti et al., 2018). With disease progression performance worsens, and emotion recognition deficits become salience and robust among manifest patients (Johnson et al., 2007, Tabrizi et al., 2009).
The reduced negative emotion recognition in HD patients is correlated with reduced grey matter volumes in the insula, orbitofrontal cortex (OFC), hippocampus (Ille et al., 2011), and different regions of the occipital lobe (Scahill et al., 2013). The insula and the OFC are associated with emotion processing (Sprengelmeyer, Rausch, Eysel, & Przuntek, 1998), while the occipital lobe is involved in tasks requiring the processing of visual stimuli (Scahill et al., 2013). However, there is conflicting evidence for association between negative emotion recognition performance and striatal atrophy (Scahill et al., 2013, Johnson et al., 2007, but see; Harrington et al., 2014, Henley et al., 2008).
Understanding emotional disturbances in HD is of particular interest. First, since the ability to recognize emotions is a key social skill, there is a strong impact of an impairment in this domain on patients' and their relatives' quality of life. Second, early identification of symptom onset can facilitate symptomatic management and other interventions that may improve patients' quality of life at the prodromal stage and at early stages of manifest HD. Medical care may include, among others, diagnostic evaluation, management of symptoms (for example, addressing early signs of chorea, sleep disturbance, depression and anxiety), and an emphasis on good nutrition and exercise (Nance, 2007). Furthermore, identification of subtle emotional and cognitive symptoms, before considerable motor signs, increases the predictability of the disease onset. This suggests that a treatment that may delay or even prevent the onset of manifest disease could be possible in the future (Ross & Tabrizi, 2011). Therefore, the interest in emotion recognition deficits as an early sign of the prodromal stage in HD gene carriers has grown increasingly over the last years. Third, from a neurobiological point of view, the associations between the behavioral deficits and the affected brain regions in HD may contribute to emotion recognition models by elucidating the underlying mechanisms and brain structures involved in the processing of facial expressions (Henley et al., 2012).
While emotion recognition deficits among gene carriers and HD patients are well documented, their underlying cognitive mechanisms remain unclear. A key question regarding these mechanisms is whether the impairment represents a specific emotion recognition deficit, or rather, if it occurs in the context of a more widespread cognitive impairment (Bäckman et al., 1997, Paulsen, 2011). Patients demonstrate deterioration of general cognition across a wide range of cognitive functions, including attention, working memory, executive functions and inhibitory control (e.g., Duff et al., 2010, Lawrence et al., 1998, Montoya et al., 2006). At the prodromal stage of the disease, a subtle cognitive decline is detectable around 10–20 years before the predicted disease onset, which is best detected by the Symbol Digit Modalities Test, a sensitive task requiring visual attention and psychomotor speed (Tabrizi et al., 2013). In general, tests with a substantial motor or psychomotor component tend to be more sensitive at the prodromal and early stages of HD, emphasizing the interaction between motor and cognitive aspects of the disease (Ross et al., 2014). Reduced performance in emotion recognition tasks may be a sequela of this cognitive deterioration.
Another possible mechanism that may underlie emotion recognition deficits in HD is related to the motor deficits in the disease. According to embodiment theories, which emphasize the link between action and perception, there is a reciprocal relationship between the bodily expression of emotion and the manner in which emotional information is interpreted (Niedenthal, 2007). Numerous studies have shown that while watching a facial expression, people subtly produce the perceived facial expression themselves (Hess & Fischer, 2013). This sensorimotor simulation of the perceived facial expression is believed to be an important source of information which facilitates emotion recognition (Wood, Rychlowska, Korb, & Niedenthal, 2016). Hence, if the motor system is impaired, one area that may be affected is the ability to perceive and decipher facial expressions (Kordsachia, Labuschagne, & Stout, 2017).
In the present study we aimed to examine emotion recognition in HD patients, using a novel approach to emotion recognition testing. The vast majority of previous studies examined emotion recognition in HD patients with static images of stereotypical and highly intense facial expressions from sets developed or inspired by the ‘Basic Emotion’ approach (Bora et al., 2016, Ekman and Friesen, 1976, Johnson et al., 2007). By contrast, our working assumption was that real-life emotional cues are dynamic, and often also subtle and non-stereotypical (Gaspar et al., 2014, Jack et al., 2014, Yitzhak et al., 2017).
Thus, in contrast to the mainstream method of examining emotion perception using static stereotypical facial expressions, our first goal was to examine emotion recognition using dynamic facial expressions that are also low in intensity and stereotypicality. To this end, we used two different facial expression datasets containing dynamic facial expressions, as facial configuration changes from neutral to emotional in a few seconds video clip. The first, consisting of intense and stereotypical emotional facial displays (ADFES: Van Der Schalk, Hawk, Fischer, & Doosje, 2011). The second, a novel dataset consisting of subtle emotional facial displays conveyed in an unconstrained manner (JeFEE: Yitzhak et al., 2017). Unlike sets in which the actors were strictly instructed and trained to activate specific facial muscles to convey stereotypical configurations (e.g., Ekman and Friesen, 1976, Van Der Schalk et al., 2011), in this set the actors had the liberty of using any facial muscles they wished. When validating this set (Yitzhak et al., 2017) it was shown that the JeFEE subtle expressions were not merely low intensity stereotypical expressions, but rather that they included different muscular cues, compared to the stereotypical expressions, when conveying the emotions. The JeFEE subtle stimuli were also rated as more ecologically looking and judged as better representations of facial expressions in every-day life, compared to the highly stereotypical stimuli (Yitzhak et al., 2017). Therefore, patients' performance in recognizing these facial expressions may be a more valid measure regarding their difficulties in real-life social interactions.
Aside from improved ecological validity, using subtle and non-stereotypical expressions may also carry the benefit of detecting slight deficits in emotion recognition. Specifically, while emotion recognition deficits are well established in symptomatic HD patients, among prodromal HD gene carriers this deficit appeared to be less pronounced and less consistently demonstrated (for a meta-analysis, see Bora et al., 2016). These findings indicate that at the prodromal period of HD, facial expression recognition is not a very sensitive measure for the detection of early symptoms, as it is at later stages of the disease. Since recognizing subtle non-stereotypical expressions is more challenging compared to the highly recognizable stereotypical stimuli, early impairments of HD may be easier to detect using such an emotion recognition task.
In line with previous studies, we predicted that HD patients' emotion recognition will be impaired compared to control participants. However, we expected this impairment to be more pronounced for the subtle, non-stereotypical stimuli in comparison to the highly intense and stereotypical stimuli.
In addition to characterizing emotion recognition in HD patients and controls, our second goal was to examine how different factors, such as cognitive status, motor symptoms, depression symptoms and an estimation of HD pathology progression, are correlated with emotion recognition decline in HD patients. Particularly, we aimed to compare two possible predictors of emotion recognition performance: general cognitive screening and severity of motor symptoms. The predictive power of these two factors on emotion recognition performance can shed light on its underling mechanism. On the one hand, if the cognitive screening scores are strongly associated with emotion recognition, it will reinforce the assumption that the reduced performance in emotion recognition task is one aspect of more general cognitive deterioration. On the other hand, if the motor symptom severity is strongly correlated with the accuracy of facial expression classification, it will reinforce the action-perception hypothesis, suggesting that emotion recognition deficits in HD patients are rooted in impaired motor control.
Section snippets
Participants
We recruited 21 HD patients from the Parkinson's and Movement Disorders Unit at the Sourasky Medical Center, Tel-Aviv, Israel. All participants were clinically diagnosed with symptomatic HD, at the manifest period of the disease. Genetic information on the number of CAG repeats was available for 19 patients. The remaining two patients had a family history of HD but they were not genetically tested.1
Emotion recognition in HD
Our first aim was to characterize emotion recognition performance of dynamic facial expressions in HD patients versus controls, and to examine the effect of different facial expression sets (intense [ADFES set] vs subtle [JeFEE set]), and different emotions (6 basic emotions) on performance. For between group comparisons of emotion recognition performance across sets and emotions, see Table 2.
A mixed ANOVA with group (HD/Control) as a between subjects factor, stimulus set (ADFES/JeFEE) and
Discussion
In accordance with previous studies, HD patients exhibited considerable impairments in facial expression recognition, compared to control participants. These results extend previous findings by demonstrating that patients' performance was affected by the characterization of the observed facial expressions. While HD patients' performance with intense and stereotypical facial expressions was impaired but remained above chance, for subtle and non-stereotypical facial expressions, their recognition
Author declaration
No part of the study procedures or analyses was pre-registered in a time-stamped, institutional registry prior to the research being conducted.
We report how we determined our sample size, all data exclusions, all inclusion/exclusion criteria, whether inclusion/exclusion criteria were established prior to data analysis, all manipulations, and all measures in the study.
Open practices
The study in this article earned Open Materials and Open Data badges for transparent practices. Data and R code for the study are available at https://doi.org/10.5281/zenodo.3634926.
The stimuli of the JeFEE set are freely available for research purposes. Readers seeking access to the stimuli are requested to fill in the online conditions web form - https://forms.gle/ix5JeCQo76SM5Z8C8. Readers seeking access to the.
ADFES stimulus set are advised to visit the Amsterdam Interdisciplinary Centre
CRediT authorship contribution statement
Neta Yitzhak: Conceptualization, Methodology, Software, Formal Analysis, Investigation, Data Curation, Writing – Original Draft, Visualization, Project Administration. Tanya Gurevich: Conceptualization, Supervision. Noit Inbar: Data Curation, Resources, Project Administration. Maya Lecker: Formal Analysis, Investigation. Doron Atias: Investigation. Hadasa Avramovich: Investigation. Hillel Aviezer: Conceptualization, Methodology, Resources, Writing – Review & Editing, Supervision, Funding
Acknowledgments
This work was supported by an EU Career Integration Grant [EU-CIG 618597] to Hillel Aviezer.
References (68)
- et al.
Social cognition in Huntington's disease: A meta-analysis
Behavioural Brain Research SreeTestContent1
(2016) - et al.
Emotion recognition in Huntington's disease: A systematic review
Neuroscience and Biobehavioral Reviews
(2012) - et al.
Defective emotion recognition in early HD is neuropsychologically and anatomically generic
Neuropsychologia
(2008) - et al.
Emotion recognition and experience in Huntington's disease: Is there a differential impairment?
Psychiatry Research
(2011) - et al.
Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time
Current Biology
(2014) - et al.
Diminished facial EMG responses to disgusting scenes and happy and fearful faces in Huntington's disease
Cortex
(2018) - et al.
Beyond emotion recognition deficits: A theory guided analysis of emotion processing in Huntington's disease
Neuroscience and Biobehavioral Reviews
(2017) - et al.
Cognitive functions and corticostriatal circuits: Insights from Huntington's disease
Trends in Cognitive Sciences
(1998) - et al.
Functional imaging of face and hand imitation: Towards a motor theory of empathy
Neuroimage
(2004) Comprehensive care in Huntington's disease: A physician's perspective
Brain Research Bulletin
(2007)
Striatal and white matter predictors of estimated diagnosis for Huntington disease
Brain Research Bulletin
Language within our grasp
Trends in Neurosciences
Huntington's disease: From molecular pathogenesis to clinical treatment
Lancet Neurology
Emotion recognition in Huntington's disease and frontotemporal dementia
Neuropsychologia
Social cognition in frontotemporal dementia and Huntington's disease
Neuropsychologia
Biological and clinical manifestations of Huntington's disease in the longitudinal TRACK-HD study: Cross-sectional analysis of baseline data
The Lancet Neurology
Predictors of phenotypic progression and disease onset in premanifest and early-stage Huntington's disease in the TRACK-HD study: Analysis of 36-month observational data
The Lancet Neurology
Joint recognition–expression impairment of facial emotions in Huntington's disease despite intact understanding of feelings
Cortex
Huntington's disease
The Lancet
Assessment of depression in medical patients: A systematic review of the utility of the Beck depression inventory-II
Clinics
Fashioning the face: Sensorimotor simulation contributes to facial expression recognition
Trends in Cognitive Sciences
Exploring emotion regulation and emotion recognition in people with presymptomatic Huntington's disease: The role of emotional awareness
Neuropsychologia
Rate of caudate atrophy in presymptomatic and symptomatic stages of Huntington's disease
Movement Disorders: Official Journal of the Movement Disorder Society
Cognitive deficits in Huntington's disease are predicted by dopaminergic PET markers and brain volumes
Brain: A Journal of Neurology
Beck depression inventory-II
San Antonio
Behavioural changes in Huntington's disease
Neuropsychiatry Neuropsychology and Behavioral Neurology
Validation of self-report depression rating scales in Huntington's disease
Movement Disorders
Consistent neurodegeneration and its association with clinical progression in Huntington's disease: A coordinate-based meta-analysis
Neuro-Degenerative Diseases
Cognitive deficits in Huntington's disease on the repeatable battery for the assessment of neuropsychological status
Journal of Clinical and Experimental Neuropsychology
Early changes in white matter pathways of the sensorimotor cortex in premanifest Huntington's disease
Human Brain Mapping
Interaction without intent: The shape of the social world in Huntington's disease
Social Cognitive and Affective Neuroscience
Pictures of facial affect
G∗Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences
Behavior Research Methods
The 'shared manifold' hypothesis. From mirror neurons to empathy
Journal of Consciousness Studies
Cited by (14)
Emotion expression through spoken language in Huntington disease
2022, CortexCitation Excerpt :Traditional methods seeking to compare the expression of emotions between patients and controls use either a statistical comparison of pre-defined spoken language markers (Alhinti et al., 2021) or a human jury (Hayes et al., 2009; Trinkler et al., 2013). These methods preclude analysing variation of everyday emotions beyond stereotypical constrained intense emotions in static set-ups (Yitzhak et al., 2020). Our methods allow assessing naturally triggered emotions (Harmon-Jones et al., 2007; Nazareth et al., 2019) without any assumptions between the speech markers and the nature of emotion.
A dynamic constraint representation approach based on cross-domain dictionary learning for expression recognition
2022, Journal of Visual Communication and Image RepresentationCitation Excerpt :Facial expression recognition (FER) has proven valuable in several application areas, including healthcare and information security [1–4].
Mood and emotional disorders associated with parkinsonism, Huntington disease, and other movement disorders
2021, Handbook of Clinical NeurologyCitation Excerpt :Impairment in the recognition of positive emotions is milder and is only apparent in symptomatic individuals (Bora et al., 2016). This finding holds true, not only with the use of traditional stimuli of static stereotypical facial expressions but also with the use of slightly more challenging paradigms employing dynamic facial expressions (Yitzhak et al., 2020). There is growing evidence of the presence of impairments in the recognition of vocal emotional prosody in patients with HD (Bora et al., 2016).
BIMODAL EMOTION DEPTH RECOGNITION METHOD OF FACIAL EXPRESSION AND POSTURE IN CYBER-PHYSICAL SYSTEMS
2024, Mechatronic Systems and Control