Skip to main content
SpringerLink
Log in

Process mining for self-regulated learning assessment in e-learning

  • Published:
Journal of Computing in Higher Education Aims and scope Submit manuscript

Abstract

Content assessment has broadly improved in e-learning scenarios in recent decades. However, the e-Learning process can give rise to a spatial and temporal gap that poses interesting challenges for assessment of not only content, but also students’ acquisition of core skills such as self-regulated learning. Our objective was to discover students’ self-regulated learning processes during an e-Learning course by using Process Mining Techniques. We applied a new algorithm in the educational domain called Inductive Miner over the interaction traces from 101 university students in a course given over one semester on the Moodle 2.0 platform. Data was extracted from the platform’s event logs with 21,629 traces in order to discover students’ self-regulation models that contribute to improving the instructional process. The Inductive Miner algorithm discovered optimal models in terms of fitness for both Pass and Fail students in this dataset, as well as models at a certain level of granularity that can be interpreted in educational terms, which are the most important achievement in model discovery. We can conclude that although students who passed did not follow the instructors’ suggestions exactly, they did follow the logic of a successful self-regulated learning process as opposed to their failing classmates. The Process Mining models also allow us to examine which specific actions the students performed, and it was particularly interesting to see a high presence of actions related to forum-supported collaborative learning in the Pass group and an absence of those in the Fail group.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Aljawarneh, S., Muhsin, Z., Nsour, A., Alkhateeb, F., & AlMaghayreh, E. (2010). E-learning tools and technologies in education: A perspective. In The fifth international conference of learning international networks consortium (LINC). Cambridge, MA: MIT. Retrieved from http://people.math.sfu.ca/~vjungic/shadi.pdf. Accessed 5 Jan 2019.

  • Azevedo, R., & Aleven, V. (Eds.). (2013). International handbook of metacognition and learning technologies. Amsterdam: Springer.

    Google Scholar 

  • Azevedo, R., & Feyzi-Behnagh, R. (2011). Dysregulated learning with advanced learning technologies. Journal of e-Learning and Knowledge Society,7(2), e9–e18.

    Google Scholar 

  • Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-Behnagh, R., Bouchet, F., et al. (2013). Using trace data to examine the complex roles of cognitive, metacognitive, and emotional self-regulatory processes during learning with multi-agent systems. In R. Azevedo & V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 427–449). New York: Springer.

    Chapter  Google Scholar 

  • Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning,9(2), 161–185. https://doi.org/10.1007/s11409-013-9107-6.

    Article  Google Scholar 

  • Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research,74(3), 379–439.

    Article  Google Scholar 

  • Biggs, J. B. (2005). Calidad del aprendizaje universitario (Quality of university learning). Madrid: Narcea.

    Google Scholar 

  • Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology,64, 417–444. https://doi.org/10.1146/annurev-psych-113011-143823.

    Article  Google Scholar 

  • Bogarín, A., Cerezo, R., & Romero, C. (2018a). A survey on educational process mining. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. https://doi.org/10.1002/widm.1230.

    Article  Google Scholar 

  • Bogarin, A., Cerezo, R., Romero, C. (2018b). Discovering learning processes using inductive miner: A case study with learning management systems (LMSs). Psicothema, 30(3), 322–329. https://doi.org/10.7334/psicothema2018.116.

    Article  Google Scholar 

  • Bogarín, A., Romero, C., Cerezo, R., & Sánchez-Santillán, M. (2014). Clustering for improving educational process mining. In M. Pistilli, J. Willis, & D. Koch (Eds.), Proceedings of the fourth international conference on learning analytics and knowledge (pp. 170–181). Indianapolis: ACM. https://doi.org/10.1145/2567574.2567604.

    Chapter  Google Scholar 

  • Buijs, J. C., Van Dongen, B. F., & van Der Aalst, W. M. (2012). On the role of fitness, precision, generalization and simplicity in process discovery. In R. Meersman, H. Panetto, T. Dillon, S. Rinderle-Ma, P. Dadam, X. Zhou, S. Pearson, A. Ferscha, S. Bergamaschi, & I. F. Cruz (Eds.), Proceedings of the OTM confederated international conferences “on the move to meaningful internet systems” (pp. 305–322). Berlin: Springer. https://doi.org/10.1007/978-3-642-33606-5_19.

    Chapter  Google Scholar 

  • Cerezo, R., Esteban, M., Sánchez-Santillán, M., & Núñez, J. C. (2017). Procrastinating behavior in computer-based learning environments to predict performance: A case study in Moodle. Frontiers in Psychology,8, 1403.

    Article  Google Scholar 

  • Cerezo, R., Fernández, E., Amieiro, N., Valle, A., Rosário, P., & Núñez, J. C. (2018). Mediating role of self-efficacy and usefulness between self-regulated learning strategy knowledge and its use. Revista de Psicodidáctica. https://doi.org/10.1016/j.psicod.2018.08.001.

    Article  Google Scholar 

  • Cerezo, R., Nuñez, J. C., Rosario, P., Valle, A., Rodriguez, S., & Bernardo, A. (2010). New Media for the promotion of self-regulated learning in higher education. Psicothema,22(2), 306–315.

    Google Scholar 

  • Cerezo, R., Romero, C., Bogarín, A., & Núñez, J.C. (2014). Improving performance and comprehensibility of educational process mining models for a better understanding of the learning process. In Metacognition 2014. 6th Bienal meeting of the EARLI Special Interest Group 16. Estambul, Turquia (pp. 1–2).

  • Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., & Núñez, J. C. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education,96, 42–54. https://doi.org/10.1016/j.compedu.2016.02.006.

    Article  Google Scholar 

  • Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. New Jersey: Wiley.

    Book  Google Scholar 

  • Commission, European. (2014). New modes of learning and teaching in higher education. Luxembourg: European Union.

    Google Scholar 

  • Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008). Internet-based learning in the health professions: A meta-analysis. JAMA,300(10), 1181–1196.

    Article  Google Scholar 

  • Dahlstrom, E., Brooks, D. C., & Bichsel, J. (2014). The current ecosystem of learning management systems in higher education: Student, faculty, and IT perspectives (Research report). Retrieved from http://www.educause.edu/ecar.2014EDUCAUSE.CCby-nc-nd. Accessed 20 Dec 2018.

  • Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge (pp. 9–17). ACM. https://doi.org/10.1145/2090116.2090118.

  • Emond, B., & Buffett, S. (2015). Analyzing student inquiry data using process discovery and sequence classification. Paper presented at the International Educational Data Mining Society, Madrid, Spain.

  • Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). The KDD process for extracting useful knowledge from volumes of data. Communications of the ACM,39(11), 27–34. https://doi.org/10.1145/240455.240464.

    Article  Google Scholar 

  • García-Peñalvo, F. J., & Seoane Pardo, A. M. (2015). Una revisión actualizada del concepto de eLearning. Education in the Knowledge Society,16, 119–144. https://doi.org/10.14201/eks2015161119144.

    Article  Google Scholar 

  • Ge, X., & Harde, P. L. (2010). Self-processes and learning environment as influences in the development of expertise in instructional design. Learning Environment Research,13(1), e23–e41.

    Article  Google Scholar 

  • Gómez-Aguilar, D. A., Hernández-García, A., García-Penñalvo, F. J., & Theron, R. (2015). Tap into visual analysis of customization of grouping of activities in eLearning. Computers in Human Behavior,47, e60–e67.

    Article  Google Scholar 

  • Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. Handbook of Self-Regulation of Learning and Performance,30, 65–84.

    Google Scholar 

  • Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Oxon: Routledge.

    Book  Google Scholar 

  • Hu, Y. H., Lo, C. L., & Shih, S. P. (2014). Developing early warning systems to predict students’ online learning performance. Computers in Human Behavior,36, 469–478. https://doi.org/10.1016/j.chb.2014.04.002.

    Article  Google Scholar 

  • Lara, J. A., Lizcano, D., Martínez, M. A., Pazos, J., & Riera, T. (2014). A system for knowledge discovery in e-learning environments within the European Higher Education Area-Application to student data from Open University of Madrid, UDIMA. Computers & Education,72, 23–36.

    Article  Google Scholar 

  • Leemans, S. J., Fahland, D., & van der Aalst, W. M. (2013). Discovering block-structured process models from event logs containing infrequent behaviour. Paper Presented at the International Conference on Business Process Management, Beijing, China.

  • Leemans, S. J., Fahland, D., & van der Aalst, W. M. (2014). Process and deviation exploration with inductive visual miner. BPM (Demos), 1295, 46.

    Google Scholar 

  • Lehmann, T., Hähnlein, I., & Ifenthaler, D. (2014). Cognitive, metacognitive and motivational perspectives on preflection in self-regulated online learning. Computers in Human Behavior,32, e313–e323. https://doi.org/10.1016/j.chb.2013.07.051.

    Article  Google Scholar 

  • Lust, G., Elen, J., & Clarebout, G. (2013). Regulation of tool-use within a blended course: Student differences and performance effects. Computers & Education,60(1), 385–395.

    Article  Google Scholar 

  • Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Jessup: US Department of Education.

    Google Scholar 

  • Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education,70, 29–40. https://doi.org/10.1016/j.compedu.2013.07.033.

    Article  Google Scholar 

  • Núñez, J. C., Cerezo, R., Bernardo, A., Rosário, P., Valle, A., Fernández, E., et al. (2011). Implementation of training programs in self-regulated learning strategies in moodle format: Results of a experience in higher education. Psicothema,23(2), e274–e281.

    Google Scholar 

  • Paechter, M., Maier, B., & Macher, D. (2010). Evaluation universitärer Lehre mittels Einschätzungen des subjektiven Kompetenzerwerbs. Psychologie in Erziehung und Unterricht,2, 128–138.

    Article  Google Scholar 

  • Reimann, P., Markauskaite, L., & Bannert, M. (2014). E-Research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology,45(3), 528–540. https://doi.org/10.1111/bjet.12146.

    Article  Google Scholar 

  • Romero, C., Cerezo, R., Bogarín, A., & Sánchez-Santillán, M. (2016). Educational process mining: a tutorial and case study using Moodle data sets. In Data mining and learning analytics: Applications in educational research (pp. 1–28). Wiley & Blackwell. https://doi.org/10.1002/9781118998205.ch1.

    Google Scholar 

  • Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers & Education,68, 458–472.

    Article  Google Scholar 

  • Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery,3(1), 12–27.

    Google Scholar 

  • Romero, C., Ventura, S., & y García, A. E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers & Education, 51(1), 368–384.

    Article  Google Scholar 

  • Sanchez-Santillan, M., Paule-Ruiz, M., Cerezo, R., & Alvarez-García, V. (2016). MeL: Modelo de adaptación dinámica del proceso de aprendizaje en eLearning. Anales de psicología,32(1), 106–114.

    Article  Google Scholar 

  • Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R., Abrami, P. C., Wade, C. A., et al. (2009). Technology’s effect on achievement in higher education: A Stage I meta-analysis of classroom applications. Journal of Computing in Higher Education,21(2), 95–109.

    Article  Google Scholar 

  • van der Aalst, W. M. (2011). Process discovery: An introduction. In W. M. P. van der Aalst, (Ed.), Process mining (pp. 125–156). Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-642-19345-3_5.

    Chapter  Google Scholar 

  • van der Aalst, W. M. (2016). Process mining: Data science in action. Berlin: Springer. https://doi.org/10.1007/978-3-662-49851-4.

    Book  Google Scholar 

  • Van der Aalst, W., Adriansyah, A., & van Dongen, B. (2012). Replaying history on process models for conformance checking and performance analysis. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery,2(2), 182–192.

    Google Scholar 

  • Wolff, A., Zdrahal, Z., Herrmannova, D., Kuzilek, J., & Hlosta, M. (2014). Developing predictive models for early detection of at-risk students on distance learning modules. In Machine learning and learning analytics workshop at the 4th international conference on learning analytics and knowledge (LAK14), 24–28 March 2014, Indianapolis, IN, USA. Retrieved from http://lak14indy.wordpress.com/. Accessed 5 Jan 2019.

  • Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational psychologist,25(1), 3–17.

    Article  Google Scholar 

  • Zimmerman, B. J. (2013). Theories of self-regulated learning and academic achievement: An overview and analysis. In B. J. Zimmerman & D. H. Schunk (Eds.), self-regulated learning and academic achievement (pp. 10–45). London: Routledge.

    Chapter  Google Scholar 

  • Zimmerman, B. J., & Schunk, D. (2011). Handbook of self-regulation of learning and performance. New York: Routledge.

    Google Scholar 

Download references

Acknowledgments

This work was funded by the Department of Science and Innovation (Spain) under the National Program for Research, Development and Innovation: project TIN2017-83445-P. We have also received funds from the European Union, through the European Regional Development Funds (ERDF); and the Principality of Asturias, through its Science, Technology and Innovation Plan FC-GRUPIN-IDI/2018/000199.

Author information

Authors and Affiliations

  1. Faculty of Psychology, University of Oviedo, Plaza Feijoo s/n Room 231, 33003, Oviedo, Asturias, Spain

    Rebeca Cerezo

  2. University of Córdoba, Córdoba, Spain

    Alejandro Bogarín & Cristóbal Romero

  3. University of Oviedo, Oviedo, Spain

    María Esteban

Authors
  1. Rebeca Cerezo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alejandro Bogarín
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. María Esteban
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cristóbal Romero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rebeca Cerezo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cerezo, R., Bogarín, A., Esteban, M. et al. Process mining for self-regulated learning assessment in e-learning. J Comput High Educ 32, 74–88 (2020). https://doi.org/10.1007/s12528-019-09225-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12528-019-09225-y

Keywords

Search

Navigation