Investigating 4D movie audiences’ emotional responses to motion effects and empathy
Introduction
Movies are experiential products that elicit various emotions in an audience (Cooper-Martin, 1991). The audience evaluates a movie based on its emotional experiences (Elisahberg & Sawhney, 1994, Shapiro & Biggers, 1987). Emotional experiences, such as gaining pleasure from watching a movie or feeling emotions that the characters express, form audiences' positive attitudes toward a movie (Choi & Lim, 2009). Emotions are a person's inner feelings and state of mind caused by external or internal factors (Kim et al., 2016). The audience experiences emotions mainly by understanding and empathizing with the characters when watching a movie (Jones & Dawkins, 2018).
The emotions that the audience feels while watching a movie may vary depending on personal characteristics such as social/cultural background, personality, and empathy (Hoffner, 2009; Kim et al., 2016; Wied et al., 1994). Empathy is the ability to accurately perceive emotional information from others (Levenson & Ruef, 1992). Empathy makes each member of the audience perceive pleasure (Hoekstra et al., 1999; Oliver, 1993) and the emotions of characters differently (Davis et al., 1987). Therefore, empathy is a key factor in observing the emotional responses of movie audiences.
The movie industry has recently introduced 4D effects in movies to enhance audiences’ emotional experiences. 4D effects are physical effects such as motion, vibration, lighting, smell, and wind, which are provided with 2D or 3D video. Most 4D movies are screened by several 4D system manufactures, such as D-BOX in Canada, MediMotion in the United States, and CJ 4DPlex in South Korea. CJ 4DPlex operates 737 4D theaters in 65 countries (CJ4DPLEX, 2020).
Among all 4D effects, the motion of the seat, called the motion effect, appears in the largest proportion (58.4%) of 4D movies (Lee, Han, & Choi, 2016a, 2016b). Motion effects are produced by imitating features such as camera movements, object movements, and sounds. Some studies have proposed authoring tools for 4D broadcasting (Kim, 2013; Kim et al., 2010; Waltl et al., 2013), automatic annotation methods (which annotate the video segment that can provide 4D effects; Abreu et al., 2019; Zhou et al., 2018), and automatic 4D effect generators by extracting audiovisual features (Lee & Choi, 2013; Lee et al., 2016a, 2016b; Shin et al., 2014). In the industry, the creators of 4D effects produce motion effects with utmost care for “story and emotion” (Kim, 2015). However, the manner in which specific motion effects affect the audience's emotional state has not been studied carefully, mostly because 4D movies have only recently become popular. Although CJ 4DPLEX debuted in 2009, it expanded globally in 2017 (CJ4DPLEX, 2020). Nevertheless, this knowledge is indispensable for designing cognitively compelling motion effects for 4D movies. To the authors' knowledge, this was the first study to examine the emotional responses of 4D movie audience when they experience motion effects.
The emphasis was on understanding the association between the audiences' emotional responses, their level of empathy and the characteristics of single pitch motions. The research questions (RQs) were as follows: RQ1. Among the single pitch motion, empathy, and movie clips, what are the factors that influence audiences’ emotional responses? RQ2. Do the emotional responses vary depending on the levels of a single pitch motion, empathy, and movie clips?
In a single pitch motion, the chair exhibits a pitch up or pitch down motion, and then slowly returns to the neutral posture. Participants were asked to watch movie clips with or without single pitch motion effects. Based on these results, some design guidelines on motion effects for eliciting the desired emotional responses from 4D movie audience were proposed.
The remainder of this paper is structured as follows. The surveys conducted for this study are described in Section 2. The experimental methods are described in detail in Section 3. The experimental results are described in Section 4. The implications of the results and future research directions are discussed in Section 5. Finally, the study is concluded in Section 6.
Section snippets
Preliminary study
Three surveys were conducted. In the first survey, an emotion taxonomy was constructed by collecting emotion-related words from literature and the Korean dictionary (subsection 2.1); in the second survey, six 4D movies were viewed and the types of motion effects and the counts of each type were examined (subsection 2.2); and, in the third survey, movie clips that provide motion effects and elicit emotions were collected (subsection 2.3).
Materials and methods
The experiment was designed as a mixed-subject design for three independent variables and 13 dependent variables. One between-subject variable was the level of empathy, and participants were divided into a high empathy group and a low empathy group (3.1.3 Empathy, 3.3 Participants) based on the average empathy quotient (EQ) score of Koreans, which was established by Jeon et al. (2018). Another between-subject variable was the single pitch motion effect (pitch; 3 levels). Participants were
Results
Statistical analyses were conducted with SPSS version 20; IBM, Armonk, NY, USA. Table 9 shows the emotional responses of the participants to each clip. The mean score of arousal for all clips was greater than 50, with ANG_AV2 showing the highest score at 70.42 and SUR_HP the lowest at 56.17. Valence was highest at 63.50 in SUR_HP and lowest at 43.12 in ANG_AV1. Participants’ highest tragic score (mean = 42.08) was for ANG_AV2, and their highest angry score (mean = 39.83) was for ANG_AV1. The
Effects that cause differences in emotional responses
The audience's emotional responses were affected by pitch, clips, and empathy; however, they were depended on the type of emotion (satisfying RQ1). There were differences in emotional responses according to the level of the single pitch motion, clips, and empathy. In the order of pitch, clips, and empathy, the differences in emotional responses to the effects and the inferred causes were described (satisfying RQ2).
The three-way ANOVA results indicated that, the intensity of surprised, worried,
Conclusions
This study further elucidates the relationships among motion effect, empathy, and emotion, and it experimentally verified the vague idea that 4D movies will affect the emotions of the audience. This study investigated the difference in emotional responses to single pitch motion effects, level of empathy of participants, and movie clips through experiments. Based on the results, five guidelines for designing motion effects in 4D movies were suggested. This study will be helpful to researchers
Credit author statement
Dawoon Jeong: Methodology, Formal analysis, Investigation, Writing – original draft, Writing – review & editing, Visualization; Sung H. Han: Conceptualization, Supervision, Writing – review & editing; Dong Yeong Jeong: Methodology, Writing – review & editing; Kimin Kwon: Investigation; Seungmoon Choi: Funding acquisition, Writing – review & editing
Funding
This work was supported by Samsung Research Funding & Incubation Center of Samsung Electronics [project number SRF-IT1802-05].
Data statement
Jeong, Dawoon; Han, Sung H. (2021), “4D movie audiences' emotional responses to motion effects and empathy”, Mendeley Data, V1, https://doi.org/10.17632/ktt5t9gsnk.1.
Declaration of competing interest
None.
Acknowledgments
None.
References (67)
- et al.
DECAF: MEG-based multimodal database for decoding affective physiological responses
IEEE Transactions on Affective Computing
(2015) - et al.
October). Semi-automatic synchronization of sensory effects in mulsemedia authoring tools
- et al.
Measuring emotion: The self-assessment manikin and the semantic differential
Journal of Behavior Therapy and Experimental Psychiatry
(1994) - et al.
The empathy quotient: An investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences
Journal of Autism and Developmental Disorders
(2004) Eliciting emotion with film: Development of a stimulus set
(2011)- et al.
The emotional movie database (EMDB): A self-report and psychophysiological study
Applied Psychophysiology and Biofeedback
(2012) - et al.
The roles of sympathy and empathy on the effects of dramatic on attitude toward film
Journal of Consumer Studies
(2009) What is 4DX: Motion Seat & Signature Effects
(2020)Consumers and movies: Some findings on experiential products
ACR North American Advances
(1991)- et al.
Emotional reactions to dramatic film stimuli: The influence of cognitive and emotional empathy
Journal of Personality and Social Psychology
(1987)
A new standardized emotional film database for Asian culture
Frontiers in Psychology
A new and faster test to assess vestibular perception
Frontiers in Neurology
Basic emotions
Handbook of cognition and emotion
Modeling goes to Hollywood: Predicting individual differences in movie enjoyment
Management Science
Narrative processing: Building consumer connections to brands
Journal of Consumer Psychology
Physiological responses induced by emotion-eliciting films
Applied Psychophysiology and Biofeedback
Vestibular-somatosensory interactions: Effects of passive whole-body rotation on somatosensory detection
PloS One
Visual-vestibular conflict detection depends on fixation
Current Biology
Emotion elicitation using films
Cognition & Emotion
Empathy across the adult lifespan: Longitudinal and experience-sampling findings
Emotion
Elapsed time estimates in virtual reality and the physical world: The role of arousal and emotional valence
Computers in Human Behavior
Evaluation of vestibular thresholds for motion detection in the Simona research simulator
A revised film set for the induction of basic emotions
Cognition & Emotion
Autobiographical memories about the experience of seeing frightening movies in childhood
Media Psychology
Affective responses and exposure to frightening films: The role of empathy and different types of content
Communication Research Reports
The development of the dizziness handicap inventory
Archives of Otolaryngology - Head and Neck Surgery
Hybrid deep neural networks for face emotion recognition
Pattern Recognition Letters
November). Identification of the optimal emotion recognition algorithm using physiological signals
The 2nd international conference on engineering and industries (ICEI), seogwipo, South Korea
Building a database of 4D movie clips eliciting affect/emotions
Is Internet overuse associated with impaired empathic ability in Korean college students?
Medicine
Walking in someone else's shoes: Creating empathy in the practice of immersive film
Media Practice and Education
Development and assessment of film excerpts used for emotion elicitation
Activitas Nervosa Superior Rediviva
Getting a grip on other minds: Mirror neurons, intention understanding, and cognitive empathy
Social Neuroscience
Cited by (5)
Feasibility Study on Enhancing User Experience of Smart Cinemas Using Internet of Things (IoT) Technology
2023, Proceedings of the 2023 IEEE 6th International Conference on Knowledge Innovation and Invention, ICKII 2023Merging Camera and Object Haptic Motion Effects for Improved 4D Experiences
2023, Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2023Behavior Psychology Learning Management of Cultural Change Conflict based on the Media Film the Last Samurai
2023, Journal for ReAttach Therapy and Developmental DiversitiesCultural Entertainment Consumption and Empathy Communication Mechanism
2022, Frontiers in PsychologyMotion Effects: Perceptual Space and Synthesis for Specific Perceptual Properties
2022, IEEE Transactions on Haptics