Identifying needs for learning analytics adoption in Latin American universities: A mixed-methods approach
Introduction
Latin American universities and colleges have an urgent need to foster student persistence and improve quality assurance (Cobo & Aguerrebere, 2018; Ferreyra, Avitabile, Botero Álvarez, Haimovich Paz, & Urzúa, 2017; Knobel & Bernasconi, 2017). To address these challenges, some researchers have suggested building capacities for Learning Analytics (LA)1 in the region, so higher education systems can use educational data to improve learning outcomes (Cobo & Aguerrebere, 2018; Lemos dos Santos, Cechinel, Carvalho Nunes, & Ochoa, 2017; Maldonado-Mahauad et al., 2018). From current practice in the UK and other developed countries, researchers have argued that LA could become a valuable strategy for boosting retention rates, reducing quality disparities, improving the allocation of resources, monitoring skill development, and increasing the number of graduates (Gasevic, 2018; Sclater, Peasgood, & Mullan, 2016). Thus, it is anticipated that LA could provide potential solutions to address similar educational challenges in Latin American and other developing countries (Gasevic, 2018; Sclater et al., 2016).
However, the early LA promise of improving learning and its environments has not been completely fulfilled. So far, there is limited evidence that demonstrates the impact of LA services on learning results, teaching and learning processes, and institutional decision-making (Ferguson et al., 2016; Gasevic, Jovanović, Pardo, & Dawson, 2017; Viberg, Hatakka, Bälter, & Mavroudi, 2018), even in regions where researchers have made more progress in the development of LA services (i.e. North America, Europe and Australia) According to Gasevic (2018), the availability and deployment of LA tools and methods does not guarantee institutional benefits if LA adoption is not closely integrated with learning design at a classroom level, and with decision-making processes at an institutional level (Gasevic, 2018). To benefit from LA adoption, some universities have started to strategically plan for leveraging educational data on a large scale, but these institutions are only a handful (Colvin, Dawson, & Fisher, 2015; Colvin, Dawson, Wade, & Gasevic, 2017).
To overcome challenges for LA adoption, researchers have proposed several frameworks and instruments to guide the deployment of LA tools and methods in Europe, North America and Australia (Colvin et al., 2017; Dawson et al., 2018). Although this work has allowed researchers to identify key dimensions that generally affect the adoption of LA services (Dawson et al., 2018), there are still complex siloes, conflicting leadership agendas, and a wide variety of issues that are specific to each institution (Zilvinskis, Willis, & Borden, 2017). In the light of the complexity of higher education contexts, the SHEILA (Supporting Higher Education to Integrate Learning Analytics) project produced a number of materials for engaging key stakeholders and formulating policies for LA adoption in European universities (Tsai et al., 2018). These materials include protocols for conducting surveys, interviews and focus groups to explore stakeholder needs for LA services, along with other aspects such as culture and existing capabilities for LA adoption. As the interest in LA grows in Latin America, these materials could be helpful if they are adapted to explore needs for LA services from the perspective of various stakeholders.
While the SHEILA protocols analysed various themes related to LA adoption (Tsai et al., 2018), this paper focuses on identifying needs of students, teaching staff, and managers from Latin American universities in terms of LA services. Considering that the maturity levels of LA adoption in Latin America fall far behind European levels, we adapted the SHEILA protocols to the local context of four Latin American universities affiliated to a large-scale project to build capacity for the design and implementation of LA tools in the region (LALA project: https://www.lalaproject.org/). As a result of adapting these protocols, we collected qualitative data by means of semi-structured interviews with 37 senior managers, and 16 focus groups with 51 teaching staff and 45 students respectively. We also collected quantitative data by conducting a student survey and a staff survey, which attracted responses from 1884 students and 368 teaching staff.
To the best of our knowledge, this paper is the first effort to provide empirical evidence about educational challenges and needs for LA adoption from the perspectives of higher education stakeholders in Latin America. First, this paper presents a literature review on current challenges in higher education systems in the region, followed by an analysis of the needs for LA tools and methods to address these challenges. Then, the paper illustrates how mixed methods were used to identify needs of each stakeholder. Based on these findings, we discussed considerations and recommendations for LA adoption in the Latin American region.
Section snippets
Literature review
Latin American universities have started to analyse teaching and learning processes through LA, but these attempts are still on a small scale (Lemos dos Santos et al., 2017). Additionally, the region lacks a community to exchange ideas, methods, and tools because of the limited availability of experienced researchers in this field (Cobo & Aguerrebere, 2018; Lemos dos Santos et al., 2017). To make a timely contribution for scaling LA adoption in Latin America, we first present an overview of
Research design and objectives
This paper is part of the first phase of a large-scale project that aims to build capacity to design and implement LA tools in Latin America (LALA project-https://www.lalaproject.org/). The research question addressed in this paper is: What are the educational needs for LA adoption in Latin American universities from the perspective of students, teaching staff and managers therein? To answer this research question, we adopted a mixed convergent-parallel approach to complement qualitative
Findings
Table 4 reports the five main educational needs for LA adoption in Latin American universities. These findings are based on the analysis of the qualitative and quantitative information collected from the three stakeholders. The following sub-sections were organized to describe the needs of each stakeholder separately.
Discussion and considerations for LA services in Latin America
By triangulating the data collected from four different Latin American universities, we identified needs for LA services that have been previously documented in literature, regardless of the fact that this literature has been generated primarily outside of this region. First, students need quality feedback and timely support to improve their academic performance (Findings 1 and 2 in Table 4), which could allow them to reflect on their learning (Bodily & Verbert, 2017; Tsai et al., 2018).
Limitations and future work
This study presents findings obtained from four flagship universities in Chile and Ecuador. These universities are relevant to the system as a whole by serving as benchmarks for student training, faculty evaluation, and research generation (Knobel & Bernasconi, 2017). However, we expect other institutions to analyse if our findings resonate with their contextual needs, in order to motivate data-driven strategies across the region. This is why similar studies and analysis should be done in other
Conclusions
This study expands current research about LA adoption in Latin America by contributing empirical evidence about the needs of different stakeholders for LA services. These needs are associated to existing processes in many universities and colleges, such as feedback provision and evaluations of teaching, which provide a convenient starting point to integrate data-driven strategies at an institutional level. It is important that institutions take into account the views of different stakeholders
Declaration of Competing Interest
None.
Acknowledgments
This work was funded by the EU LALA project (grant no. 586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP). This project has been funded with support from the European Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. The authors would like to thank the reviewers for their constructive suggestions, in addition to the Laspau fellowship program, organization affiliated to
References (37)
- et al.
The current landscape of learning analytics in higher education
Computers in Human Behavior
(2018) Ecuador: Students Take to Streets Against Education Austerity Measures
- et al.
Learning analytics methods, benefits, and challenges in higher education: A systematic literature review
Online Learning Journal
(2016) - et al.
Higher education reforms: Latin America in comparative perspective
Education Policy Analysis Archives
(2017) - et al.
Review of research on student-facing learning analytics dashboards and educational recommender systems
IEEE Transactions on Learning Technologies
(2017) - et al.
Social learning analytics
Journal of Education Technology and Society
(2012) - et al.
Sistema de Acreditación Universitaria en Chile: ¿Cuánto hemos avanzado? [university accreditation system in Chile: How much have we advanced?]
Estudios Pedagogicos
(2014) - et al.
Building capacity for learning analytics in Latin America
- et al.
Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement
(2015) - et al.
Addressing the challenges of institutional adoption