Benefits of additional online practice opportunities in higher education

https://doi.org/10.1016/j.iheduc.2021.100834Get rights and content

Highlights

  • We analyzed the effect of additional online practice opportunities on exam points.

  • Additional practice included self-testing, spacing, and direct feedback.

  • University students in a math course were observed during a semester and not in a lab.

  • Practice participation and performance led to better outcomes on the final exam.

  • Results are robust to a rich set of control variables.

Abstract

Are exam grades predetermined by students' prior performance and personal characteristics, or can underperforming students catch up? We evaluate whether additional e-learning practice opportunities improve learning outcomes for a group of undergraduate business students enrolled in a university math course (N = 281). During the semester, students were offered two types of voluntary additional e-learning practice opportunities (some earned extra credit, others did not). These practice opportunities incorporated the study techniques of self-testing and spacing, as well as knowledge of correct responses feedback. After controlling for a large number of personal characteristics, we find that voluntary practice has a statistically significant effect on exam performance, which indicates that practicing leads to better grades. Our results show that students currently performing at any level can improve their learning outcomes through additional practice. Furthermore, the overall effect is most significant for weak students who would otherwise be expected to score low on the exam.

Introduction

E-learning practice opportunities can mitigate challenges of teaching higher education courses (Ardac & Sezen, 2002; Dusi & Huisman, 2021). When instructors design their courses, they can offer only a limited set of study materials that promote a selected set of study techniques; including too many materials will overwhelm students. Instructors face particularly narrow limitations when teaching heterogenous first-year introductory courses with a large number of students. E-learning allows instructors to offer practice materials that indirectly promote a variety of study techniques. These techniques have different effects on students' learning gains, but students (and instructors) might not know which techniques are most effective (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013; Karpicke, 2016). Dunlosky et al. (2013) review articles on evidence-based study techniques and points out the minimal learning gains of rereading, highlighting, and summarizing. However, these are the study techniques that most students use most frequently (Karpicke, 2016). Teaching students how to use other study techniques takes time and resources; furthermore, this instruction might not reach all students in courses where attendance is not mandatory. Therefore, it is essential to know whether e-learning practice can be used to augment a course so that students can apply more effective study techniques without explicit instruction in those techniques. Furthermore, while intrinsically motivated students will take on additional tasks (Walker, Greene, & Mansell, 2006), the additional e-learning materials should be designed to help all students in a course, particularly underperforming students. Therefore, the general objective of the present study is to analyze whether and which students benefit from additional e-learning opportunities that are based on effective study techniques.

A rich body of literature highlights the study techniques of self-testing (also called active recall) and spacing (also called spaced learning), which outperform rereading, highlighting, and summarizing (Dunlosky et al., 2013; Hartwig & Dunlosky, 2012; Rawson & Dunlosky, 2012; Rodriguez, Fischer, Zhou, Warschauer, & Massimelli, 2021; Roediger & Pyc, 2012). Moreover, additional e-learning practice opportunities could be designed to provide students with immediate feedback about the correct response. Especially in classes that are too large for instructors to provide individual feedback, automated feedback is helpful (Butler, Karpicke, & Roediger III, 2008; Finn, Thomas, & Rawson, 2018). However, previous studies have mixed findings on whether additional practice opportunities incorporated within the design of a course can improve students' course achievement (B. W. Brown & Liedholm, 2002; Panus, Stewart, Hagemeier, Thigpen, & Brooks, 2014). Therefore, more research on blended learning is needed to see if there are effective ways to provide students with practice opportunities and whether previous findings on study techniques and feedback can be translated into real-life higher education settings.

In this paper, we analyze the effect of a course augmented with e-learning practice opportunities. We observed a first-year college mathematics course with 280 undergraduate students. Students enrolled in the course were either majoring or minoring in business administration or economics at a large public German university. Therefore, we used a real-life educational setting, which is important to establish the external validity of laboratory results (Morrison & Anglin, 2005; Ross & Morrison, 1989). In addition, the study goes beyond existing results by identifying which students benefitted the most from e-learning practice opportunities. In particular, we seek to determine whether e-learning practice only helps students who are already doing well or whether additional digital practice opportunities can help underperforming students catch up.

Section snippets

Literature review

The study relates to three strands of existing literature: study techniques (self-testing and spacing), knowledge of correct response feedback, and blended learning. In addition, to select an appropriate set of control variables, we surveyed the literature in these areas, which identifies essential predictors/drivers of students' learning and achievement.

Aims of the study

Most literature on blended learning compares students who are taught in different formats, but few previous studies have explicitly studied the effects of specific features in a blended format. Though the objective of Panus et al. (2014) is similar to that of our study, their analysis focuses on multiple-choice quizzes. Our study aimed to test the effectiveness of two different e-learning practice opportunities on high-level learning in a higher education mathematics course. In addition, we

Design of online exercises and participants

Mathematics for Economics and Business Administration is a compulsory module in the first semester of all bachelor's degree programs in economics and business administration (majors and minors) at the German public university where the study was conducted. The course has three voluntary practice tests, which are offered through the built-in test feature in the university's open-source online learning management system (ILIAS). These practice tests are designed primarily to give students the

Regression results

Table 3 presents the regression results for the baseline and full model presented in Eq. (1). The full model includes the covariates identified via double selection OLS using the LASSO, random forest, xgBoost, or all available covariates. The outcome variable is the standardized final exam score without the extra credit points students received for completing the practice tests. While the complete model might have too many control variables to allow sufficient degrees of freedom for making

Discussion

In this section, we discuss the mechanisms and channels through which students might have benefited from the practice options. We also explore the effect of including control variables when measuring the effect of practice. In addition, we discuss potential reasons for the differences between submissions on the MAD app and practice exam attempts. Finally, we comment on the results of the quantile regressions.

Conclusion

We evaluated two types of voluntary e-learning opportunities offered to students taking a mathematics lecture course designed for first-year business administration and economics students. The first practice option consisted of three online practice tests with repeated access and extra credit for the first attempt on a specific date; the second consisted of the “Matrix a Day” app. We find that students who took the optional practice tests in the e-learning environment generally improved their

Author note

Jakob Schwerter is an affiliate at the LEAD Graduate School & Research Network [GSC1028], funded by the Excellence Initiative of the German federal and state governments.

This work was partly funded by Jacobs Foundation Advanced Research Fellowship to Kou Murayama. This research was partly supported by JSPS KAKENHI (Grant Numbers 16H06406)” and the “Alexander von Humboldt Foundation (the Alexander von Humboldt Professorship endowed by the German Federal Ministry of Education and Research)“

References (103)

  • T. Joyce et al.

    Does classroom time matter?

    Economics of Education Review

    (2015)
  • Y. Karlen et al.

    The role of implicit theories in students' grit, achievement goals, intrinsic and extrinsic motivation, and achievement in the context a long-term challenging task

    Learning and Individual Differences

    (2019)
  • M. Komarraju et al.

    Role of the big five personality traits in predicting college students' academic motivation and achievement

    Learning and Individual Differences

    (2009)
  • M. Komarraju et al.

    The big five personality traits, learning styles, and academic achievement

    Personality and Individual Differences

    (2011)
  • A.W. Melton

    The situation with respect to the spacing of repetitions and memory

    Journal of Verbal Learning and Verbal Behavior

    (1970)
  • P.R. Pintrich

    An achievement goal theory perspective on issues in motivation terminology, theory, and research

    Contemporary Educational Psychology

    (2000)
  • H.L. Roediger et al.

    Inexpensive techniques to improve education: Applying cognitive psychology to enhance educational practice

    Journal of Applied Research in Memory and Cognition

    (2012)
  • C.O. Walker et al.

    Identification with academics, intrinsic/extrinsic motivation, and self-efficacy as predictors of cognitive engagement

    Learning and Individual Differences

    (2006)
  • W.T. Alpert et al.

    A randomized assessment of online learning

    American Economic Review: Papers and Proceedings

    (2016)
  • D. Ardac et al.

    Effectiveness of computer-based chemistry instruction in enhancing the learning of content and variable control under guided versus unguided conditions

    Journal of Science Education and Technology

    (2002)
  • R. Azevedo

    Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning

    Educational Psychologist

    (2005)
  • T.H. Bailey et al.

    The influence of motivation and adaptation on students' subjective well-being, meaning in life and academic performance

    Higher Education Research and Development

    (2016)
  • R. Baker et al.

    The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: Opening the black box of learning processes

    International Journal of Educational Technology in Higher Education

    (2020)
  • R.L. Bangert-Drowns et al.

    The instructional effect of feedback in test-like events

    Review of Educational Research

    (1991)
  • L.E. Baranik et al.

    Examining the construct validity of mastery-avoidance achievement goals: A meta-analysis

    Human Performance

    (2010)
  • A. Becker et al.

    The relationship between economic preferences and psychological personality measures

    Annual Review of Economics

    (2012)
  • A. Belloni et al.

    Inference on treatment effects after selection among high-dimensional controls

    Review of Economic Studies

    (2014)
  • R.A. Bjork

    Retrieval as a memory modifier: An interpretation of negative recency and related phenomena

  • B.W. Brown et al.

    Can web courses replace the classroom in principles of microeconomics?

    American Economic Review

    (2002)
  • A.C. Butler

    Repeated testing produces superior transfer of learning relative to repeated studying

    Journal of Experimental Psychology

    (2010)
  • A.C. Butler et al.

    Correcting a metacognitive error: Feedback increases retention of low-confidence correct responses

    Journal of Experimental Psychology. Learning, Memory, and Cognition

    (2008)
  • A.C. Butler et al.

    Correcting a metacognitive error: Feedback increases retention of low-confidence correct responses

    Journal of Experimental Psychology. Learning, Memory, and Cognition

    (2008)
  • C.M. Clark et al.

    When and why introducing difficulties and errors can enhance instruction

  • D. Clark et al.

    Using goals to motivate college students: Theory and evidence from field experiments

    Review of Economics and Statistics

    (2019)
  • W.L. Cull

    Untangling the benefits of multiple study opportunities and repeated testing for cued recall

    Applied Cognitive Psychology

    (2000)
  • P. Denny et al.

    Empirical support for a causal relationship between gamification and learning outcomes

    CHI

    (2018)
  • J.M. Digman

    Personality structure: Emergence if the five-factor model

    Annual Review of Psychology

    (1990)
  • S.R. Dobrow et al.

    Managing the grading paradox: Leveraging the power of choice in the classroom

    Academy of Management 2010 Annual Meeting - Dare to Care: Passion and Compassion in Management Practice and Research, AOM 2010

    (2010)
  • J. Dunlosky et al.

    Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology

    Psychological Science in the Public Interest, Supplement

    (2013)
  • D. Dusi et al.

    It's more complex than it seems! Employing the concept of prosumption to grasp the heterogeneity and complexity of student roles in higher education

    Higher Education

    (2021)
  • H. Ebbinghaus

    Über das Gedächtnis: Untersuchungen zur experimentellen Psychologie (memory: A contribution to experimental psychology)

    (1885)
  • J.S. Eccles et al.

    Expectancies, values, and academic behaviors

  • A.J. Elliot et al.

    A 2 × 2 achievement goal framework

    Journal of Personality and Social Psychology

    (2001)
  • A.J. Elliot et al.

    Achievement goals, study strategies, and exam performance: A mediational analysis

    Journal of Educational Psychology

    (1999)
  • A.J. Elliot et al.

    On the measurement of achievement goals: Critique, illustration, and application

    Journal of Educational Psychology

    (2008)
  • K.M. Ericson

    On the interaction of memory and procrastination: Implications for reminders, deadlines, and empirical estimation

    European Economic Association

    (2017)
  • N.T. Feather

    Values, valences, expectations, and actions

    Journal of Social Issues

    (1992)
  • S. Frederick et al.

    Time discounting and time preference : A critical review

    Journal of Economic Literature

    (2002)
  • C. Garaus et al.

    The hidden power of small rewards: The effects of insufficient external rewards on autonomous motivation to learn

    Academy of Management Learning & Education

    (2016)
  • L.R. Goldberg

    The structure of phenotypic personality traits

    American Psychologist

    (1993)
  • View full text