Identifying needs for learning analytics adoption in Latin American universities: A mixed-methods approach

https://doi.org/10.1016/j.iheduc.2020.100726Get rights and content

Highlights

  • The adoption of Learning Analytics (LA) services is still incipient in Higher Education Institutions in Latin America

  • We identified needs for LA in Latin American universities from the perspectives of students, teaching staff, and managers

  • Study findings highlight areas of interest in LA including quality feedback and timely data-driven support for students

  • It is believed that LA can provide quality and actionable information to inform educational interventions

Abstract

Learning Analytics (LA) is perceived to be a promising strategy to tackle persisting educational challenges in Latin America, such as quality disparities and high dropout rates. However, Latin American universities have fallen behind in LA adoption compared to institutions in other regions. To understand stakeholders' needs for LA services, this study used mixed methods to collect data in four Latin American Universities. Qualitative data was obtained from 37 interviews with managers and 16 focus groups with 51 teaching staff and 45 students, whereas quantitative data was obtained from surveys answered by 1884 students and 368 teaching staff. According to the triangulation of both types of evidence, we found that (1) students need quality feedback and timely support, (2) teaching staff need timely alerts and meaningful performance evaluations, and (3) managers need quality information to implement support interventions. Thus, LA offers an opportunity to integrate data-driven decision-making in existing tasks.

Introduction

Latin American universities and colleges have an urgent need to foster student persistence and improve quality assurance (Cobo & Aguerrebere, 2018; Ferreyra, Avitabile, Botero Álvarez, Haimovich Paz, & Urzúa, 2017; Knobel & Bernasconi, 2017). To address these challenges, some researchers have suggested building capacities for Learning Analytics (LA)1 in the region, so higher education systems can use educational data to improve learning outcomes (Cobo & Aguerrebere, 2018; Lemos dos Santos, Cechinel, Carvalho Nunes, & Ochoa, 2017; Maldonado-Mahauad et al., 2018). From current practice in the UK and other developed countries, researchers have argued that LA could become a valuable strategy for boosting retention rates, reducing quality disparities, improving the allocation of resources, monitoring skill development, and increasing the number of graduates (Gasevic, 2018; Sclater, Peasgood, & Mullan, 2016). Thus, it is anticipated that LA could provide potential solutions to address similar educational challenges in Latin American and other developing countries (Gasevic, 2018; Sclater et al., 2016).

However, the early LA promise of improving learning and its environments has not been completely fulfilled. So far, there is limited evidence that demonstrates the impact of LA services on learning results, teaching and learning processes, and institutional decision-making (Ferguson et al., 2016; Gasevic, Jovanović, Pardo, & Dawson, 2017; Viberg, Hatakka, Bälter, & Mavroudi, 2018), even in regions where researchers have made more progress in the development of LA services (i.e. North America, Europe and Australia) According to Gasevic (2018), the availability and deployment of LA tools and methods does not guarantee institutional benefits if LA adoption is not closely integrated with learning design at a classroom level, and with decision-making processes at an institutional level (Gasevic, 2018). To benefit from LA adoption, some universities have started to strategically plan for leveraging educational data on a large scale, but these institutions are only a handful (Colvin, Dawson, & Fisher, 2015; Colvin, Dawson, Wade, & Gasevic, 2017).

To overcome challenges for LA adoption, researchers have proposed several frameworks and instruments to guide the deployment of LA tools and methods in Europe, North America and Australia (Colvin et al., 2017; Dawson et al., 2018). Although this work has allowed researchers to identify key dimensions that generally affect the adoption of LA services (Dawson et al., 2018), there are still complex siloes, conflicting leadership agendas, and a wide variety of issues that are specific to each institution (Zilvinskis, Willis, & Borden, 2017). In the light of the complexity of higher education contexts, the SHEILA (Supporting Higher Education to Integrate Learning Analytics) project produced a number of materials for engaging key stakeholders and formulating policies for LA adoption in European universities (Tsai et al., 2018). These materials include protocols for conducting surveys, interviews and focus groups to explore stakeholder needs for LA services, along with other aspects such as culture and existing capabilities for LA adoption. As the interest in LA grows in Latin America, these materials could be helpful if they are adapted to explore needs for LA services from the perspective of various stakeholders.

While the SHEILA protocols analysed various themes related to LA adoption (Tsai et al., 2018), this paper focuses on identifying needs of students, teaching staff, and managers from Latin American universities in terms of LA services. Considering that the maturity levels of LA adoption in Latin America fall far behind European levels, we adapted the SHEILA protocols to the local context of four Latin American universities affiliated to a large-scale project to build capacity for the design and implementation of LA tools in the region (LALA project: https://www.lalaproject.org/). As a result of adapting these protocols, we collected qualitative data by means of semi-structured interviews with 37 senior managers, and 16 focus groups with 51 teaching staff and 45 students respectively. We also collected quantitative data by conducting a student survey and a staff survey, which attracted responses from 1884 students and 368 teaching staff.

To the best of our knowledge, this paper is the first effort to provide empirical evidence about educational challenges and needs for LA adoption from the perspectives of higher education stakeholders in Latin America. First, this paper presents a literature review on current challenges in higher education systems in the region, followed by an analysis of the needs for LA tools and methods to address these challenges. Then, the paper illustrates how mixed methods were used to identify needs of each stakeholder. Based on these findings, we discussed considerations and recommendations for LA adoption in the Latin American region.

Section snippets

Literature review

Latin American universities have started to analyse teaching and learning processes through LA, but these attempts are still on a small scale (Lemos dos Santos et al., 2017). Additionally, the region lacks a community to exchange ideas, methods, and tools because of the limited availability of experienced researchers in this field (Cobo & Aguerrebere, 2018; Lemos dos Santos et al., 2017). To make a timely contribution for scaling LA adoption in Latin America, we first present an overview of

Research design and objectives

This paper is part of the first phase of a large-scale project that aims to build capacity to design and implement LA tools in Latin America (LALA project-https://www.lalaproject.org/). The research question addressed in this paper is: What are the educational needs for LA adoption in Latin American universities from the perspective of students, teaching staff and managers therein? To answer this research question, we adopted a mixed convergent-parallel approach to complement qualitative

Findings

Table 4 reports the five main educational needs for LA adoption in Latin American universities. These findings are based on the analysis of the qualitative and quantitative information collected from the three stakeholders. The following sub-sections were organized to describe the needs of each stakeholder separately.

Discussion and considerations for LA services in Latin America

By triangulating the data collected from four different Latin American universities, we identified needs for LA services that have been previously documented in literature, regardless of the fact that this literature has been generated primarily outside of this region. First, students need quality feedback and timely support to improve their academic performance (Findings 1 and 2 in Table 4), which could allow them to reflect on their learning (Bodily & Verbert, 2017; Tsai et al., 2018).

Limitations and future work

This study presents findings obtained from four flagship universities in Chile and Ecuador. These universities are relevant to the system as a whole by serving as benchmarks for student training, faculty evaluation, and research generation (Knobel & Bernasconi, 2017). However, we expect other institutions to analyse if our findings resonate with their contextual needs, in order to motivate data-driven strategies across the region. This is why similar studies and analysis should be done in other

Conclusions

This study expands current research about LA adoption in Latin America by contributing empirical evidence about the needs of different stakeholders for LA services. These needs are associated to existing processes in many universities and colleges, such as feedback provision and evaluations of teaching, which provide a convenient starting point to integrate data-driven strategies at an institutional level. It is important that institutions take into account the views of different stakeholders

Declaration of Competing Interest

None.

Acknowledgments

This work was funded by the EU LALA project (grant no. 586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP). This project has been funded with support from the European Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. The authors would like to thank the reviewers for their constructive suggestions, in addition to the Laspau fellowship program, organization affiliated to

References (37)

  • O. Viberg et al.

    The current landscape of learning analytics in higher education

    Computers in Human Behavior

    (2018)
  • Telesur

    Ecuador: Students Take to Streets Against Education Austerity Measures

  • J.T. Avella et al.

    Learning analytics methods, benefits, and challenges in higher education: A systematic literature review

    Online Learning Journal

    (2016)
  • A. Bernasconi et al.

    Higher education reforms: Latin America in comparative perspective

    Education Policy Analysis Archives

    (2017)
  • R. Bodily et al.

    Review of research on student-facing learning analytics dashboards and educational recommender systems

    IEEE Transactions on Learning Technologies

    (2017)
  • S. Buckingham et al.

    Social learning analytics

    Journal of Education Technology and Society

    (2012)
  • V. Cancino et al.

    Sistema de Acreditación Universitaria en Chile: ¿Cuánto hemos avanzado? [university accreditation system in Chile: How much have we advanced?]

    Estudios Pedagogicos

    (2014)
  • C. Cobo et al.

    Building capacity for learning analytics in Latin America

  • C. Colvin et al.

    Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement

    (2015)
  • C. Colvin et al.

    Addressing the challenges of institutional adoption

  • J.W. Creswell

    Educational research: Planning, conducting, and evaluating quantitative and qualitative research

  • S. Dawson et al.

    Rethinking learning analytics adoption through complexity leadership theory

  • C. Edenfield

    Higher education in Ecuador: A reflective analysis

    Georgia Journal of College Student Affairs

    (2016)
  • R. Ferguson et al.

    Research evidence on the use of learning analytics: implications for education policy

  • F.H.G. Ferreira et al.

    Economic mobility and the rise of the Latin American middle class

    (2013)
  • M.M. Ferreyra et al.

    At a Crossroads: Higher Education in Latin America and the Caribbean

    (2017)
  • D. Gasevic

    Directions for adoption of learning analytics in the global south

  • D. Gasevic et al.

    Detecting learning strategies with analytics: Links with self-reported measures and academic performance

    Journal of Learning Analytics

    (2017)
  • Cited by (0)

    View full text