The effects of using Facebook as a discussion forum in an online principles of economics course: Results of a randomized controlled trial

https://doi.org/10.1016/j.iree.2019.100157Get rights and content

Abstract

This paper makes a comparison between using social media or traditional Course Management System (CMS) discussion groups in a fully online Principles of Microeconomics course. Students were randomly assigned to a discussion forum in either Facebook or CMS to discern a difference in the level of engagement and learning outcomes. The popular hypothesis is that students using social media have greater engagement with the class and higher learning outcomes relative to students using a CMS platform. Reasons for the positive effect include the ease of use and student familiarity with social media, which allows them to make more connections and gain a deeper understanding of the course material through discussions with others. Contrary to the widely held view, the results of this study suggest that participation in the treatment Facebook group is associated with declining student engagement and a reduction in the semester course average of 3 to 5 points on a 100-point scale.

Introduction

Online-only classes are growing in popularity and enrollment, now up to 32% online as a percent of total U.S. enrollment (Allen and Seaman, 2013), The growth is attributed to their asynchronous and anywhere characteristics, which allow both students and instructors maximum flexibility, and their effect of reducing the delivery cost of courses (Deming et al., 2015). the disadvantages of online courses compared to traditional instruction are a lack of connection between the instructor and the students and among the students themselves, reduced learning outcomes (Alpert et al., 2016; Joyce et al., 2015; Figlio et al., 2010), and lower retention rates (Allione and Stein, 2016; Hart, 2012). As an attempt to replace the classroom dynamic and increase learner engagement, online delivery tools (Course Management Systems or CMS) have incorporated discussion boards and messaging capabilities. However, the familiarity and accessibility of CMS pale in comparison to that of social media such as Facebook (FB). The latter has been used longer and more often by students, as well as being more accessible on mobile platforms.1 Consequently, social media has an advantage of not forcing students into an additional unfamiliar electronic portal such as a CMS.

While CMS portals do have an advantage in ease of assigning groups and managing student submissions, most of these advantages accrue to the instructor rather than the student, who only sees an additional portal that must be managed and attended. Of course, social media is not as secure or complete a platform for managing online course delivery as CMS, so grades and other FERPA-regulated content must remain in the CMS. Thus, the online student would still be required to use multiple portals. Despite this additional requirement, the hope has been that students with easier and more consistent access to social media would be more engaged in discussion forums that are hosted on social media, and that this would translate into better learning outcomes for them as a group.

The learning curve for using social media as a discussion forum is flat for students. Because most students have a social media account as well as apps on their mobile devices, there is no uncertainty or delay in accessing the discussions. This will be the case for more and more instructors as well. But even for others not yet immersed in social media, it is no more difficult to learn than a CMS interface. Messages sent on social media are read and responded to more quickly than on CMS because social media accounts are checked more frequently Al-Bahrani et al. (2015)2, and messages are sent with push notifications or alerts. Frequency of checking and push notifications are not attributes of the standard CMS. See Bosch (2009) for a thorough discussion of the advantages of Facebook as an instructional tool.

As a tradeoff to ease of access, privacy concerns must be considered when interacting with students via social media. Built-in privacy protections should be set with care, and students should be reminded of the recommended privacy settings, as they might not consider these settings. This study created a “secret” Facebook group, which added a protective layer by allowing access only to those invited into the group, while others on the social media platform could not see the group or its posts. Students were added to the group as “members,” not as “friends,” so that student privacy settings confine the instructor to the student profile in the more restrictive “public” instead of the less restrictive “friend” level. This approach addresses the “creepy tree house” downside of using social media in an instructional setting described by McBride (2008).

Most studies of the use of Facebook in an educational setting report a positive association between Facebook usage as an instructional aid and learning outcomes. (Celik et al., 2015; Anin et al., 2015; Deng and Tavares, 2013; Kent, 2013; Stack, 2013; Pellizzari, 2012; and Kabilan et al., 2010). Further, Junco (2012) found a significantly greater increase in engagement and learning outcomes when using Twitter as a discussion forum, which could be attributable to forcing students into a shorter format (Kassens, 2014). On the other hand, a few studies report that increased time spent on Facebook is associated with reduced learning outcomes (Kirschner and Karpinski, 2010; Junco, 2012 and Irwin et al., 2012). None of the prior studies uses a randomized trial design and is therefore subject to selection bias and the resulting potential mismeasurement due to confounding effects.

The randomized controlled trial (RCT) experimental design minimizes selection bias and generates consistent controls. RCTs reduce confounding bias in the allocation of covariates, apart from the one specifically controlled, and thus afford greater reliability and generalizability. The RCT methodology has yet to be applied widely in this setting (social media engagement and outcome effects), but it has begun to be used is assessing the effect of instructional format on learning outcomes (Figlio et al., 2010; US Department of Education, 2010; Rimm-Kaufman et al., 2014; Joyce et al., 2015; and Alpert et al., 2016).

This study employs a randomized controlled trial design to the use of Facebook as a discussion forum, with students randomly assigned to Facebook (treatment) or traditional CMS (control) discussion groups. Discussion threads are weekly graded assignments for students in both groups (the same assignment, at the same time, with the same rules, only the location of the group is varied). The grade (6% of the course grade) is predicated on the student’s contribution to the discussion (e.g., posing a course-related question, responding to a posted question, posting and describing a link to relevant material, or commenting on the linked material). This study addresses the differential effect social media tools used in instruction have on student engagement (as measured by the length, frequency, and quality of student posts in both groups) and learning outcomes (measured by the course grade with appropriate controls). The results of this study suggest that, relative to CMS, student discussions in social media are shorter but frequency of posts are similar. However, the discussions in social media are correlated with lower content engagement and reduced learning outcomes. These findings contradict those of most studies about the benefits of social media as an instructional tool.

Section snippets

Literature review

The use of social medial in education is receiving increased attention as “digital natives,”— those who grew up with internet and now social media—enter institutions of higher learning. The ubiquity of social media—90% of college students use Facebook daily (Smith and Caruso, 2010)—and its acceptance by instructors (see Al-Bahrani et al., 2017 for a survey of instructors) are driving new uses of social media in the classroom for the purpose of attracting, engaging, and reaching students where

Data & methodology

The data on student characteristics, engagement, and learning outcomes was collected from four sections6 of Principles of Microeconomics taught in Winter/Summer Intersessions in 2013 and 2014. Students were randomly assigned7 to discussion groups either in Facebook (treatment) or in the CMS (control) at the start of the courses. There were four FB and

Measuring the effect on engagement and learning outcomes

Engagement is measured by participation in the discussion forums as separated by Facebook (treatment) and CMS (control); four dimensions of engagement are considered: the total words posted, the average length of posts, the number of posts, and the average engagement rating of the post. Table 3 shows no significant difference across treatment and control for the number of posts or the average engagement rating of the posts, but engagement as assessed by total words posted and the average length

Student engagement

First, we use Total Words posted as a measure of student engagement. One hypothesis is that CMS encourages longer posts suggesting that more thought and preparation was put into each post, and more effort was expended as a whole. Additional effort of this nature indicates a student who is spending more time with the material and is, therefore, internalizing it more than other students lacking this type of engagement. Along the same lines as this hypothesis, FB users tend to engage in casual

Summary

The use of online courses is burgeoning, and in this milieu, educational institutions widely use CMS systems that have built-in tools for email and discussion boards. The CMS systems have advantages for the instructor in ease of assigning students to groups, as well as tools for tracking and grading student posts. The disadvantages of the CMS discussion system are on the student side: the time to access the board is slowed by logon verification procedures, resulting in students spending much

References (42)

  • Sulaiman Anin et al.

    Facebook usage, socialization and academic performance, Computers &

    Education

    (2015)
  • D.E.F. Baird et al.

    Neomillennial user experience design strategies: Utilizing social networking media to support “always on” learning styles

    J. Educ. Technol. Syst.

    (2005)
  • D. Baird et al.

    Neomillennial user experience design strategies: utilizing social networking media to support ‘Always On’ learning styles

    J. Educ. Technol. Syst.

    (2006)
  • T.E. Bosch

    Using online social networking for teaching and learning: facebook use at the University of Cape Town

    Commun.: South Afr. J. Commun. Theory and Research

    (2009)
  • Ismail Celik et al.

    A model for understanding educational facebook use

    EURSIA J. Math. Sci. Technol. Educ.

    (2015)
  • H. Coates

    A model of online and general campus‐based student engagement

    Assess. Eval. High. Educ.

    (2007)
  • D.J. Deming et al.

    Can online learning bend the higher education cost curve?

    Am. Econ. Rev.

    (2015)
  • D.N. Figlio et al.

    Is It Live or Is It Internet?

    (2010)
  • O.R. Harmon et al.

    Online discussion and learning outcomes

    Int. Adv. Econ. Res.

    (2013)
  • C. Hart

    Factors associated with student persistence in an online program of study: a review of the literature

    J. Inter. Online Learn.

    (2012)
  • Christopher Irwin et al.

    Students’ perceptions of using facebook as an interactive learning resource at university

    Australasian J. Educ.Technol.

    (2012)
  • Cited by (12)

    • Teaching with Twitter: An extension to the traditional learning environment.

      2022, International Review of Economics Education
      Citation Excerpt :

      Arulampalam et al. (2012) study heterogeneity in attendance effects, Ng et al. (2011) analyse learning styles, and Deerfield (2019) study of co-operative learning. More recently, Harmon, Tomolonis (2019) consider how observing the impact of Facebook use at different levels of attainment may inform robustness of past research in social media use. In each case there is power in looking beyond first order moments such as the mean.

    • Not for want of trying: Effort and Success of women in principles of microeconomics

      2020, International Review of Economics Education
      Citation Excerpt :

      Not all effort is the same. Harmon and Tomolonis (2019) find that participation in social media-based discussion forums decreases exam performance in Principles of Micro. Self (2013) differentiates between required and voluntary effort on out-of-class assignments, reporting that voluntary completion of problem sets is positively associated with improved exam performance while completing required assignments has no effect.

    • Evaluating the effects of mobile applications on course assessment: A quasi-experiment on a macroeconomics course

      2020, International Review of Economics Education
      Citation Excerpt :

      This paper examines the relation between assessment and the use of mobile technologies as SRS in the classroom. Contrary to the general expectations, the results of this study show that mobile learning applications have no positive effects on assessment in the short-term, which concurs with the findings of Chui et al. (2013) and Harmon and Tomolonis (2019). However, positive attitudes towards engaging with new technologies, technologies in the classroom and, specifically, favourable towards Socrative, were found (concurring with Guarascio et al., 2017; Lim, 2017; Zou and Lambert, 2017).

    View all citing articles on Scopus
    View full text