Beyond positive and negative emotions: Looking into the role of achievement emotions in discussion forums of MOOCs
Introduction
With the recent development in open educational resources in both industry and academia, Massive Open Online Courses (MOOCs) have taken the center stage of discussion especially in the higher education sector (Reich, 2015; Wautelet et al., 2016). MOOCs can enable thousands of students to take courses at their convenience without cost or at low cost. It is largely because they provide a specific means for enabling more equitable access to learning that MOOCs have gained popularity in recent years (Tang and Carr-Chellman, 2016). In spite of all their potential, a significant concern about MOOCs is the extremely high attrition rate (approximately 90%) that has been reported (Hew & Cheung, 2014). Such a high dropout rate has often been cited as a scale-efficacy tradeoff (Onah, Sinclair, & Boyatt, 2014). While the reasons for dropping out are diverse in MOOCs, studies have shown that students' emotions evidenced in their learning process can significantly affect students' continued participation (Dillon et al., 2016; Dillon et al., 2016; Hillaire, Iniesto, & Rienties, 2017).
Understanding the role of emotion becomes more important especially if we consider the gradual nature of attrition in MOOCs. Much of the research on MOOC dropouts centers on the summative metric of attribution, e.g., through conducting correlational analysis between dropouts with click-stream evidence of engagement (Ramesh, Goldwasser, Huang, Daumé III, & Getoor, 2013) or building dropout prediction models (Halawa, Greene, & Mitchell, 2014). However, several seminal works have shown that attrition takes place over time (Whitehill, Mohan, Seaton, Rosen, & Tingley, 2017; Yang, Wen, Howley, Kraut, & Rose, 2015). That is, while many participants either never engage in the course at all or drop out after the first week, a significant portion of participants remains in the course for several weeks and then drops out along the way (Yang, Sinha, Adamson, & Rosé, 2013). This suggests that there are students struggling to stay involved. Understanding the participation of struggling students as they struggle and ultimately quit the course can help find potential ways to provide appropriate intervention and scaffolds to support them. Investigations on struggling students from an emotional perspective are essential to understand their participation and learning experience. Emotions have been constantly identified as one of the key factors in influencing online learning commitment (Lee & Choi, 2011; Pillay, Irving, & Tones, 2007). Supporting these struggling students from the emotional angle may be the first low hanging fruit to improve the retention rate of a MOOC.
Emotion has a complex influence on learning and commitment. Positive emotion experienced during the learning process is not necessarily related with longer commitment in MOOCs and negative emotion may lead to a beneficial effect on learning outcomes (Barak, Watted, & Haick, 2016; Lee & Choi, 2011). In fact, emotions are more than a simple dichotomy of positive and negative emotions. In learning and academic settings, the kind of emotions that really matters to the commitment and outcomes are achievement emotions (Pekrun, 2006). Achievement emotion can be further deconstructed as positive activating, positive deactivating, negative activating and negative deactivating emotions according to the control-value theory (Pekrun, 2006). As an integrative framework to approach students' learning experiences, the control-value theory of achievement emotions may serve as the basis for instructors to design interventions to improve learning engagement in MOOCs. By contrast, research previously investigated the influence of the single dimension of emotions (positive or negative emotions) on attrition overtime (Dillon, Ambrose, et al., 2016; Dillon, Bosch, et al., 2016), without revealing the mechanism for different achievement emotions on student attrition.
This work focused on exploring achievement emotions and their complex impact on student dropouts in MOOCs. It aimed to understand how achievement emotions influence learner survival as they struggle and ultimately withdraw from a MOOC forum. Relying on an actual MOOC dataset, we first depicted a classifier that automatically predicted the different achievement emotions of students' posts in the MOOC forum. Then with reasonable accuracy, the built classification model was applied to all the posts to identify the four achievement emotions. Finally, a survival modeling technique was used to quantify the effect of different achievement emotions on student attrition longitudinally. The rest of the paper is organized as follows: we begin by discussing the literature and theoretical framework underlying this study. Next, we describe the dataset and the methodology used. Then, our analysis and results are presented. Finally, we discuss the results and summarize this study by anchoring it to the prior work.
Section snippets
Dropout analysis in MOOCs
The hallmark of MOOCs is that enrolled learners are involved in the social learning with virtual unlimited peers (Reich, 2015). Participation in the course is the prerequisite for learners to benefit from the large-scale social learning. Conversely, the large number of attritions becomes the tradeoff to extend educational resources to the masses (Onah et al., 2014). Therefore, improving the retention rate of MOOCs has become a major focus of recent research.
Many studies explored the
Research dataset and context
The dataset used in this research was derived from a Coursera MOOC offered by a large public research university in the United States. This course was selected because 1) this was a highly valued MOOC by many online portals and students; 2) its course design (e.g., course format, course length, discussion forum settings) and student population (e.g., number, profile) was exemplary (Bonafini, Chae, Park, & Jablokow, 2017). With that being said, empirical findings resulting from this dataset were
Achievement emotion detection results
Using different features and various algorithms, machine learning models were built to automatically identify different kinds of achievement emotions in MOOC forum posts. Table 3 shows the results for the performance of the machine learning models. By comparing the ROC_AUC and Kappa values, SVM has the highest performance with ROC_AUC (0.91) and Kappa (0.61) when using all the language summary features, linguistic features, and the LDA topic features. The prediction performance of the machine
Discussion and conclusion
Increasing the retention rates of MOOCs is tremendously important for their future adoption. In this study, we looked beyond the dichotomy of only positive and negative emotions, using an integrative framework of achievement emotions to examine their influence on students' commitment in MOOCs. An accurate machine learning model was built to automatically detect the achievement emotions in the forum posts. Then survival analysis was applied to quantify the effect of achievement emotions on
Declaration of Competing Interest
The authors declare that they have no conflict of interests.
Wanli Xing is an Assistant Professor in Instructional Technology at Texas Tech University, USA with background in learning sciences, statistics, computer science and mathematical modeling. His research interests are educational data mining, learning analytics, and CSCL.
References (54)
- et al.
Understanding the MOOCs continuance: The role of openness and reputation
Computers & Education
(2015) - et al.
Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning
The Internet and Higher Education
(2012) - et al.
Motivation to learn in massive open online courses: Examining aspects of language and social engagement
Computers & Education
(2016) The role of emotional labor in explaining teachers' enthusiasm and students' outcomes: A multilevel mediational analysis
Learning and Individual Differences
(2019)- et al.
Students' and instructors' use of massive open online courses (MOOCs): Motivations and challenges
Educational Research Review
(2014) - et al.
Temporal predication of dropouts in MOOCs: Reaching the low hanging fruit through stacking generalization
Computers in human behavior
(2016) - et al.
Quantifying the Effect of Informational Support on Membership Retention in Online Communities through Large-Scale Data Analytics
Computers in Human Behavior
(2018) - et al.
Turn on, tune in, drop out: Anticipating student dropouts in massive open online courses
- et al.
Sensor-free automated detection of affect in a cognitive tutor for algebra
The ripple effect: Emotional contagion and its influence on group behavior
Administrative Science Quarterly
(2002)
Why does affect matter in organizations?
Academy of Management Perspectives
How much does student engagement with videos and forums in a MOOC affect their achievement
Online Learning Journal
Emotional and informational support messages in an online hospice support community
CIN: Computers, Informatics, Nursing
Affect detection: An interdisciplinary review of models, methods, and their applications
IEEE Transactions on Affective Computing
Facilitating Students’ Interaction in MOOCs through Timeline-Anchored Discussion
International Journal of Human-Computer Interaction
Automatic text classification: A technical review
International Journal of Computer Applications
A longitudinal analysis of achievement goals: From affective antecedents to emotional effects and achievement outcomes
Journal of Educational Psychology
Student affect during learning with a MOOC
Student emotion, co-occurrence, and dropout in a MOOC context
Dropout prediction in MOOCs using learner activity features
I'm sad you're sad: Emotional contagion in CMC
Toward emotionally accessible massive open online courses (MOOCs)
The link between achievement emotions, appraisals, and task performance: Pedagogical considerations for emotions in CBLEs
Journal of Computers in Education
A review of machine learning algorithms for text-documents classification
Journal of Advances in Information Technology
Survival analysis: Techniques for censored and truncated data
The challenges to connectivist learning on open online networks: Learning experiences during a massive open online course
The International Review of Research in Open and Distributed Learning
Supervised machine learning: A review of classification techniques
Cited by (76)
Is comparison the thief of joy? Students’ emotions after socially comparing their task grades, influence on their motivation
2023, International Journal of Management EducationFrom the learner's perspective: A systematic review of MOOC learner experiences (2008–2021)
2022, Computers and EducationCitation Excerpt :An earlier work by Hew (2016) provided a list of strategies that students found engaging, including active learning and peer interaction (p. 328), elements the integration of various technological tools is designed to facilitate. In other instances, technological tools can track elements of learners' engagement in MOOCs (Sun et al., 2020; Xing et al., 2019) and facilitate learners' hesitancy to interact in MOOCs. Bozkurt et al. (2018) discussed the use of a bot in MOOCs to help connect learners, and several learners learners responded to positively, a finding in keeping with another item on the list of what students find engaging in courses: instructor accessibility (Hew, 2016).
Mining opinions on LMOOCs: Sentiment and content analyses of Chinese students’ comments in discussion forums
2022, SystemCitation Excerpt :This indicates that MOOCs have acquired a significant position in China's higher education. The prominent roles of MOOCs have attracted increased research attention, which mainly centers on several areas: types of MOOCs (Downes, 2012; Hew et al., 2018; Teixeira & Mota, 2014), the impact of MOOCs on institutions (Lombardi, 2013; Motzo & Proudfoot, 2017; O'Connor, 2014), instructors' motivation and challenges in teaching MOOCs (Deng et al., 2017; Hew & Cheung, 2014), students' motivation to register MOOCs, factors influencing course completion, and perceptions of MOOCs (Deng et al., 2019; Hew & Cheung, 2014; Li et al., 2022; Wang et al., 2021; Wen et al., 2014; Xing et al., 2019), and analysis of log data of MOOCs (de Barba et al., 2016; Liu et al., 2018; Wen et al., 2014). MOOCs are generally classified into cMOOCs and xMOOCs (Downes, 2012; Teixeira & Mota, 2014).
Cognitive and emotional engagement while learning with VR: The perspective of multimodal methodology
2022, Computers and EducationContinuance Intention to use MOOCs: The Effects of Psychological Stimuli and Emotions
2024, Asia-Pacific Education Researcher
Wanli Xing is an Assistant Professor in Instructional Technology at Texas Tech University, USA with background in learning sciences, statistics, computer science and mathematical modeling. His research interests are educational data mining, learning analytics, and CSCL.
Hengtao Tang is an Assistant Professor in Educational Technology at the University of South Carolina. His research interests include learning analytics, CSCL, self-regulated learning, STEM education, and knowledge structure.
Bo Pei is a doctoral student in Instructional Technology at Texas Tech University, USA. His research interests are educational data mining, learning analytics, and CSCL.