Suggestions for Improving the Assessment of a Learning Management System Used for Clinical Curriculum Development [Letter]

Dear editor

Pinilla et al explore the use of Educational Design Research to develop a Learning Management System (LMS), an online platform for self-regulated learning activities and clinical curriculum mapping.¹ As fifth year medical students in the UK, we see the importance of online platforms to provide a structured and comprehensive learning experience. The work of Pinilla et al is of particular relevance in the context of the COVID-19 pandemic, where ward-based clinical learning has been limited, resulting in a greater reliance on online teaching modalities. Furthermore, centralised learning resources can also bridge the discrepancies in teaching quality, which students may experience in different clinical settings.

The authors’ use of student satisfaction scores to evaluate the implementation of an LMS into their curriculum is a commendable approach to understanding students’ views. However, we believe that assessing student satisfaction alone is insufficient to come to the conclusion that LMS can support student learning. Johnson et al explored both student satisfaction and academic performance to assess student learning following implementation of an LMS.² Therefore, further outcomes should be investigated to gain a better insight into the impact of LMS on students’ learning. Furthermore, students may use an LMS in different ways to support their learning - this is valuable information to inform further development of an LMS. Back et al found 63% of the student cohort used their LMS to prepare for exams.³ Thus, Pinilla et al might benefit from exploring in what manner LMS is being used to supplement students’ learning.

While the authors conclude student satisfaction was improved by the implementation of LMS, we feel the use of Likert scale-based surveys in isolation questions the credibility of this conclusion. Although Likert scales are widely used in research, respondents tend to answer in a more agreeable way to statements, thus demonstrating a susceptibility to acquiescence bias.⁴ This is an inherent and often unavoidable flaw in their use. Moreover, Sullivan et al suggest that applying descriptive statistics, such as calculating a mean, to findings from Likert scales can result in ambiguous conclusions.⁵ Pinilla et al report a significant increase in overall student satisfaction from 3.9 to 4.4. However, in the context of the Likert scale, these values seem arbitrary. We, therefore, suggest that the results from Likert scale should be used in context with other data, or a different approach to analyzing the results should be applied.

Lastly, the authors could gain from providing clear inclusion and exclusion criteria. Pinilla et al report that LMS usage time ranged from 9 to 165 minutes per student, suggesting some participants were not fully engaged with their LMS. Therefore, the findings may not be entirely representative of the cohort, potentially impacting the validity of their conclusions.

Incorporating LMS into curriculum development has the potential to be highly beneficial to medical students’ learning. We are aware that this is a study in its early phase, and we hope that we have provided some useful suggestions.