Elsevier

Computers & Education

Volume 155, October 2020, 103933
Computers & Education

Learning analytics in European higher education—Trends and barriers

https://doi.org/10.1016/j.compedu.2020.103933Get rights and content

Highlights

  • Teaching and support staff are the main users of learning analytics (LA).

  • Managers prioritise using LA to influence institutional and teaching decisions.

  • There is little evidence of active engagement with students.

  • LA is primarily implemented at small scales in European higher education.

  • Few institutions have a dedicated strategy, policy, or evaluation framework for LA.

Abstract

Learning analytics (LA) as a research field has grown rapidly over the last decade. However, adoption of LA is mostly found to be small in scale and isolated at the instructor level. This paper presents an exploratory study on institutional approaches to LA in European higher education and discusses prominent challenges that impede LA from reaching its potential. Based on a series of consultations with senior managers from 83 different higher education institutions in 24 European countries, we observe that LA is primarily perceived as a tool to enhance teaching and institutional management. As a result, teaching and support staff are found to be the main users of LA and the target audience of training support. In contrast, there is little evidence of active engagement with students or using LA to develop self-regulated learning skills. We highlight the importance of grounding LA in learning sciences and including students as a key stakeholder in the design and implementation of LA. This paper contributes to our understanding of the development of LA in European higher education and highlights areas to address in both practice and research.

Introduction

Learning analytics (LA) as a field emerged a decade ago in response to the digitalisation of education and the maturity of data mining technology (Ferguson, 2012). Under the growing pressure of financial sustainability and competition with the global market, the higher education (HE) sector is driven to demonstrate evidence of quality educational offerings. As a result, LA has risen as a means to measure learning and answer difficult questions pertaining to the overall performance of an institution in the HE sector (Viberg, Hatakka, Bälter, & Mavroudi, 2018). Despite the growing interest in using data analytics to inform educational decisions and personalise support for students, the sector has struggled to establish the value and impact of LA on the improvement of learning (Ferguson and Clow, 2017, Viberg et al., 2018). In the recent NMC Horizon Report, adaptive learning as a key objective of LA has fallen out of the list of key development areas in HE after being featured for four consecutive years (Alexander et al., 2019). In light of the trends of educational technology development and deployment, the report argues that adaptive learning technology has not been able to scale up to its potential due to various challenges in institutional adoption (Alexander et al., 2019). Our paper responds to this observation by outlining the trends in and barriers to LA adoption in the European HE sector. Our intention is to provide insights into shaping the practice and research in the field as it moves into a new decade. This paper attempts to answer the research question:

What is the state of the art in terms of learning analytics adoption in European higher education?

Drawing on survey and interview data collected in a large-scale study, we present detailed analyses of the observed phenomena, and reflect on the implications of how LA has been conceptualised and applied. In particular, we identify gaps in the roles of teachers and students in the adoption process. The study presented in this paper is by far the largest in terms of the geographical coverage, as opposed to similar studies of its kind in the same region (Ferguson et al., 2016, Nouri et al., 2019). The paper contributes to our understanding of the complex issues that impede LA from scaling, provides concrete cases illustrating the approaches taken by higher education institutions (HEIs) to move technological innovations into operation, and challenges researchers and practitioners to reflect on where we are with LA and areas to improve in order to scale the potential of LA.

Section snippets

Learning analytics in higher education

Learning Analytics (LA) is commonly defined as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs” (Long, Siemens, Conole, & Gašević, 2011). Essentially, LA makes use of the data footprints produced by students when interacting with digital technologies in the learning process for the purpose of leveraging human decisions such as designing educational

Methodology

This study adopts mixed methods using a survey and interviews. The former was primarily distributed through the European University Association (EUA) to 249 HEIs (from 38 countries in Europe) that had previously responded to an e-learning survey conducted by EUA regarding institutional experiences in e-learning (Gaebel, Kupriyanova, Morais, & Colucci, 2014). We further promoted the survey via newsletters of European-wide professional networks such as European University Information Systems

Adoption experience

The majority of the institutions had less than three years of experience adopting LA. As Fig. 1, Fig. 2 show, only 9 out of 46 institutions that participated in the interviews and 7 out of 45 institutions that responded to the survey had more than three years of experience. In terms of the scope, we labelled institutions by ‘full’(institution-wide implementation), ‘partial’ (implementation at piloting scales or in parts of the institution), ‘preparation’ (in preparation to implement LA), and

Discussion

This exploratory study consulted HEI leaders to understand the state of LA adoption in European HE. Although the study suffers from a self-selection bias, i.e., institutions that had taken interest in LA were more likely to respond to our survey and interview invitations, it is clear that the uptake of LA was at an early stage where the implementation among the interviewed institutions was primarily at small scales and few institutions had a dedicated strategy, policy, or evaluation framework

Final remarks

LA promises to enhance education by providing insights that may otherwise not be obtainable without the availability of data and technology today. The main question that concerns us is, has the intervention of LA really enhanced learning, teaching, and the overall educational environment? How should we evaluate the impact and develop our capacity to continuously learn and mature from the process of exploring the big question? In this paper, we have provided a glimpse of the current state of art

Limitations

This paper aims to present a picture of the institutional adoption of LA in European HE. To this end, we focus on our consultations with senior managers in order to take advantage of their knowledge regarding the strategic decisions and actions related to LA. As a result, this paper is limited in the diversity of perspectives, and thus should be compared with the results of our wider study (blinded for review). As mentioned previously, the study presented in this paper suffers from

CRediT authorship contribution statement

Yi-Shan Tsai: Methodology, Formal analysis, Investigation, Data curation, Writing - original draft, Writing - review & editing, Project administration. Diego Rates: Formal analysis, Investigation, Writing - original draft, Writing - review & editing. Pedro Manuel Moreno-Marcos: Formal analysis, Investigation, Writing - original draft, Writing - review & editing. Pedro J. Muñoz-Merino: Formal analysis, Investigation, Supervision, Writing - review & editing. Ioana Jivet: Formal analysis,

Acknowledgements

This work was supported by the Erasmus+ Programme of the European Union [562080-EPP- 1-2015-1-BE-EPPKA3-PI-FORWARD]. The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission will not be held responsible for any use which may be made of the information contained therein. We would like to thank the participant of this study for their generous contributions.

References (72)

  • KirschnerP.A. et al.

    The myths of the digital native and the multitasker

    Teaching and Teacher Education

    (2017)
  • SchumacherC. et al.

    Features students really expect from learning analytics

    Computers in Human Behavior

    (2018)
  • VibergO. et al.

    The current landscape of learning analytics in higher education

    Computers in Human Behavior

    (2018)
  • AlexanderB. et al.

    EDUCAUSE horizon report 2019 higher education editionTech. Rep.

    (2019)
  • Arnold, K. E., Lonn, S., & Pistilli, M. D. (2014). An exercise in institutional reflection: The learning analytics...
  • ArrowayP. et al.

    Learning analytics in higher educationTech. Rep.

    (2016)
  • AvellaJ.T. et al.

    Learning analytics methods, benefits, and challenges in higher education: A systematic literature review

    Online Learning

    (2016)
  • BrownM.G. et al.

    Conceptualizing co-enrollment: Accounting for student experiences across the curriculum

  • CarlessD. et al.

    The development of student feedback literacy: enabling uptake of feedback

    Assessment & Evaluation in Higher Education

    (2018)
  • CarlessD. et al.

    Developing sustainable feedback practices

    Studies in Higher Education

    (2011)
  • ColvinC. et al.

    Student retention and learning analytics: A snapshot of Australian practices and a framework for advancementTech. Rep.

    (2016)
  • CorrinL. et al.

    Enhancing learning analytics by understanding the needs of teachers

  • DavisF.D.

    Perceived usefulness, perceived ease of use, and user acceptance of information technology

    MIS Quarterly

    (1989)
  • DawsonS. et al.

    Increasing the impact of learning analytics

  • Dawson, S., Poquet, O., Colvin, C., Rogers, T., Pardo, A., & Gaševic, D. (2018). Rethinking learning analytics adoption...
  • Dollinger, M., & Lodge, J. M. (2018). Co-creation strategies for learning analytics. In Proceedings of the 8th...
  • DrachslerH. et al.

    Privacy and analytics: it’s a DELICATE issue a checklist for trusted learning analytics

  • El AlfyS. et al.

    Exploring the benefits and challenges of learning analytics in higher education institutions: a systematic literature review

    Information Discovery and Delivery

    (2019)
  • FergusonR.

    Learning analytics: drivers, developments and challenges

    International Journal of Technology Enhanced Learning

    (2012)
  • FergusonR. et al.

    Research evidence on the use of learning analytics: Implications for education policyTech. Rep.

    (2016)
  • Ferguson, R., & Clow, D. (2017). Where is the evidence? A call to action for learning analytics. In Proceedings of the...
  • GaebelM. et al.

    E-learning in European higher education institutions: Results of a mapping survey conducted in october-december 2013.

    European University Association

    (2014)
  • GaševićD. et al.

    Let’s not forget: Learning analytics are about learning

    TechTrends

    (2015)
  • GaševićD. et al.

    Piecing the learning analytics puzzle: a consolidated model of a field of research and practice

    Learning: Research and Practice

    (2017)
  • GaševićD. et al.

    How do we start? directions for learning analytics adoption in higher education

    International Journal of Information and Learning Technology

    (2019)
  • GrellerW. et al.

    Translating learning into numbers: A generic framework for learning analytics

    Journal of Educational Technology & Society

    (2012)
  • HendersonM. et al.

    The challenges of feedback in higher education

    Assessment & Evaluation in Higher Education

    (2019)
  • HerodotouC. et al.

    A large-scale implementation of predictive learning analytics in higher education: the teachers’ role and perspective

    Educational Technology Research and Development

    (2019)
  • HerodotouC. et al.

    Predictive learning analytics ‘at scale’: Towards guidelines to successful implementation in higher education based on the case of the open university UK

    Journal of Learning Analytics

    (2019)
  • HolsteinK. et al.

    Co-designing a real-time classroom orchestration tool to support teacher–AI complementarity

    Journal of Learning Analytics

    (2019)
  • IfenthalerD. et al.

    Higher education experts’ views on learning analytics policy recommendations for supporting study success: A german case

  • JivetI. et al.

    License to evaluate: preparing learning analytics dashboards for educational practice

  • JohnsonL. et al.

    NMC horizon report: 2016 higher education edition

    (2016)
  • Kitto, K., Shum, S. B., & Gibson, A. (2018). Embracing imperfection in learning analytics. In Proceedings of the 8th...
  • LeitnerP. et al.

    Learning analytics challenges to overcome in higher education institutions

  • LeitnerP. et al.

    Learning analytics in higher education–a literature review

  • Cited by (0)

    This document is the results of the research project funded by the Erasmus+ Programme of the European Union [562080-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD].

    1

    Present address: Goethe Universität, Robert-Mayer-Str. 11–15, 60629 Frankfurt am Main, Germany.

    2

    Present address: Monash University, 25 Exhibition Walk, Clayton, VIC 3800, Australia.

    View full text