Skip to main content

Advertisement

SpringerLink
Log in

Design review of MOOCs: application of e-learning design principles

  • Published:
Journal of Computing in Higher Education Aims and scope Submit manuscript

Abstract

The purpose of this study is to explore the pedagogical design of massive open online courses (MOOCs) using evidence-based e-learning principles. MOOCs have become an important part of discourse in higher education. However, there has been shared concern on the quality of MOOCs as learning systems for engaging learners as well as fulfilling their needs. The researchers conducted a design review of 40 computer science MOOCs from two major MOOC providers. The findings indicate a relatively low application of the principles in general, with the exception of those related to the organization and presentation of content. MOOC platforms and the difficulty level of MOOCs used the application of e-learning principles and guidelines differently. Implications for future research and design of MOOCs are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.

    Google Scholar 

  • Barak, M., Watted, A., & Haick, H. (2016). Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Computers & Education, 94, 49–60. https://doi.org/10.1016/j.compedu.2015.11.010.

    Article  Google Scholar 

  • Beaven, T., Hauck, M., Comas-Quinn, A., Lewis, T., & de los Arcos, B. (2014). MOOCs: Striking the right balance between facilitation and self-determination. Journal of Online Learning and Teaching, 10(1), 31–43.

    Google Scholar 

  • Burd, E. L., Smith, S. P., & Reisman, S. (2015). Exploring business models for MOOCs in Higher Education. Innovative Higher Education, 40, 37–49.

    Article  Google Scholar 

  • Castaño-Muñoz, J., Kreijns, K., Kalz, M., & Punie, Y. (2017). Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey. Journal of Computing in Higher Education, 29, 28–46.

    Article  Google Scholar 

  • Chen, C.-M., & Wu, C.-H. (2015). Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance. Computers & Education, 80, 108–121. https://doi.org/10.1016/j.compedu.2014.08.015.

    Article  Google Scholar 

  • Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 3, 7.

    Google Scholar 

  • Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. (2013). The MOOC phenomenon: Who takes massive open online courses and why? Retrieved January 7, 2018 from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2350964.

  • Chu, H.-C., & Hwang, G.-J. (2010). Development of a project-based cooperative learning environment for computer programming courses. International Journal of Innovation and Learning, 8(3), 256–266.

    Article  Google Scholar 

  • Chukwuemeka, E. J., Yoila, A. O., & Iscioglu, E. (2015). Instructional design quality: An evaluation of Open Education Europa Networks’ open courses using the first principles of instruction. International Journal of Science and Research, 4(11), 878–884.

    Google Scholar 

  • Clark, R. C., & Mayer, R. E. (2011). E-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: John Wiley & Sons.

    Book  Google Scholar 

  • Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–222.

    Article  Google Scholar 

  • Creelman, A., Ehlers, U., & Ossiannilsson, E. (2014). Perspectives on MOOC quality: An account of the EFQUEL MOOC quality project. The International Journal for Innovation and Quality in Learning, 2(3), 78–87.

    Google Scholar 

  • Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5, 194–197.

    Article  Google Scholar 

  • Dillahunt, T., Wang, Z., & Teasley, S. D. (2014). Democratizing higher education: Exploring MOOC use among those who cannot afford formal education. The International Review of Research in Open and Distance Learning. https://doi.org/10.19173/irrodl.v15i5.1841.

    Article  Google Scholar 

  • Egloffstein, M., & Ifenthaler, D. (2017). Employee perspectives on MOOCs for workplace learning. TechTrends, 61(1), 65–67.

    Article  Google Scholar 

  • Gonzalez, G. (2006). A systematic approach to active and cooperative learning in CS1 and its effects on CS2. In Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education (SIGCSE’06) (pp. 133–137). New York, NY: ACM Press.

  • Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.

    Article  Google Scholar 

  • Gruba, P., & Sondergaard, H. (2001). A constructivist approach to communication skills instruction in computer science. Computer Science Education, 11(3), 203–219. https://doi.org/10.1076/csed.11.3.203.3833.

    Article  Google Scholar 

  • Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the First ACM Conference on Learning @ Scale Conference (pp. 41–50). New York, NY: ACM Press.

  • Hathaway, K. L. (2013). An application of the seven principles of good practice to online courses. Research in Higher Education Journal, 22, 1.

    Google Scholar 

  • Hew, K. F. (2018). Unpacking the strategies of ten highly rated MOOCs: Implications for engaging students in large online courses. Teachers College Record, 120, 1–40.

    Google Scholar 

  • Howarth, J., D’Alessandro, S., Johnson, L., & White, L. (2017). MOOCs to university: A consumer goal and marketing perspective. Journal of Marketing for Higher Education, 27(1), 144–158.

    Article  Google Scholar 

  • Jansen, D., Rosewell, J., & Kear, K. (2017). Quality frameworks for MOOCs. In M. J. Kinshuk & M. K. Khribi (Eds.), Open education: From OERs to MOOCs (pp. 261–281). Berlin: Springer.

    Chapter  Google Scholar 

  • Konrad, A. (2017). Coursera fights to keep the promise of MOOCs alive with corporate customer push. Forbes. Retrieved January 7, 2018 from https://www.forbes.com/sites/alexkonrad/2017/12/20/coursera-goes-corporate-to-keep-alive-promise-of-moocs/#40181021543c.

  • Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563–575.

    Article  Google Scholar 

  • Linn, M., & Dalbey, J. (1989). Cognitive consequences of programming instruction [reprinted]. In E. Soloway & J. C. Spohrer (Eds.), Studying the novice programmer (pp. 57–81). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Littlejohn, A., Hooda, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. Internet and Higher Education, 29, 40–48.

    Article  Google Scholar 

  • Lowenthal, P., & Hodges, C. (2015). In search of quality: Using quality matters to analyze the quality of massive, open, online courses (MOOCs). The International Review of Research in Open and Distributed Learning, 16(5), 83–101. https://doi.org/10.19173/irrodl.v16i5.2348.

    Article  Google Scholar 

  • Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of Massive Open Online Courses (MOOCs). Computers & Education, 80, 77–83.

    Article  Google Scholar 

  • Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press.

    Book  Google Scholar 

  • Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59.

    Article  Google Scholar 

  • Milligan, C., & Littlejohn, A. (2014). Supporting professional learning in a Massive Open Online Course. The International Review of Research in Open and Distance Learning. https://doi.org/10.19173/irrodl.v15i5.1855.

    Article  Google Scholar 

  • National Academies of Sciences, Engineering, and Medicine. (2018). Assessing and responding to the growth of computer science undergraduate enrollments. Washington, D.C.: National Academies Press.

    Google Scholar 

  • Ossiannilsson, E., Williams, K., Camilleri, A. F., & Brown, M. (2015). Quality models in online and open education around the globe: State of the art and recommendations. Oslo: International Council for Open and Distance Education.

    Google Scholar 

  • Quality Matters. (2014). Introduction to the quality matters program. Retrieved January 10, 2018 from https://www.qualitymatters.org/sites/default/files/Introduction%20to%20the%20Quality%20Matters%20Program%20HyperlinkedFinal2014.pdf.

  • Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2), 137–172.

    Article  Google Scholar 

  • Rohs, M., & Ganz, M. (2015). MOOCs and the claim of education for all: A disillusion by empirical data. International Review of Research in Open and Distributed Learning., 16(6), 5. https://doi.org/10.19173/irrodl.v16i6.2033.

    Article  Google Scholar 

  • Rosewell, J., & Jansen, D. (2014). The OpenupEd quality label: Benchmarks for MOOCs. INNOQUAL: The International Journal for Innovation and Quality in Learning, 2(3), 88–100.

    Google Scholar 

  • Sanchez-Gordon, S., & Luján-Mora, S. (2018). Technological innovations in large-scale teaching: Five roots of massive open online courses. Journal of Educational Computing Research, 56(5), 623–644. https://doi.org/10.1177/0735633117727597.

    Article  Google Scholar 

  • Scagnoli, N. I., Choo, J., & Tian, J. (2019). Students’ insights on the use of video lectures in online classes. British Journal of Educational Technology, 50(1), 399–414. https://doi.org/10.1111/bjet.12572.

    Article  Google Scholar 

  • Shah, D. (2017). A product at every price: A review of MOOC stats and trends in 2017. Class Central. Retrieved January 7, 2018 from https://www.class-central.com/report/moocs-stats-and-trends-2017/.

  • Spector, J. M. (2014). Remarks on MOOCs and Mini-MOOCs. Educational Technology and Research Development, 62, 385–392.

    Article  Google Scholar 

  • Stracke, C. M. (2019). Quality frameworks and learning design for open education. International Review of Research in Open and Distributed Learning, 20(2), 180–203.

    Article  Google Scholar 

  • Tawfik, A. A., Reeves, T. D., Stich, A. E., Gill, A., Hong, C., McDade, J., et al. (2017). The nature and level of learner-learner interaction in a chemistry massive open online course (MOOC). Journal of Computing in Higher Education, 29(3), 411–431. https://doi.org/10.1007/s12528-017-9135-3.

    Article  Google Scholar 

  • Tirrell, T., & Quick, D. (2012). Chickering’s seven principles of good practice: Student attrition in community college online courses. Community College Journal of Research and Practice, 36(8), 580–590.

    Article  Google Scholar 

  • Toven-Lindsey, B., Rhoads, R. A., & Lozano, J. B. (2015). Virtually unlimited classrooms: Pedagogical practices in massive open online courses. Internet and Higher Education, 24, 1–12.

    Article  Google Scholar 

  • Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v17i2.2448.

    Article  Google Scholar 

  • Waltz, C. F., Strickland, O., & Lenz, E. R. (2010). Measurement in nursing and health research (5th ed.). New York, NY: Springer.

    Google Scholar 

  • Waters, J. (2015). How nanodegrees are disrupting higher education, campus technology. Retrieved January 11, 2018 from http://campustechnology.com/articles/2015/08/05/how-nanodegrees-are-disrupting-higher-education.aspx.

  • Watson, S. L., Loizzo, J., Watson, W. R., Mueller, C., Lim, J., & Ertmer, P. A. (2016). Instructional design, facilitation, and perceived learning outcomes: An exploratory case study of a human trafficking MOOC for attitudinal change. Educational Technology Research and Development, 64, 1273–1300.

    Article  Google Scholar 

  • Watson, W. R., Watson, S. L., & Janakiraman, S. (2017). Instructional quality of massive open online courses: A review of attitudinal change MOOCs. International Journal of Learning Technology, 12(3), 219–224.

    Article  Google Scholar 

  • Wexler, E. (2015). Moocs are still rising at least in numbers. The Chronicle of Higher Education. Retrieved January 7, 2018 from http://www.chronicle.com/blogs/wiredcampus/moocs-are-still-rising-at-least-in-numbers/57527.

  • Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Addison-Wesley Professional.

    Google Scholar 

  • Wing, J. (2011). Research notebook: Computational thinking—What and why? The Link Magazine, 6. Retrieved January 11, 2018 from https://www.cs.cmu.edu/link/research-notebook-computational-thinking-what-and-why.

  • Yang, M., Shao, Z., Liu, Q., & Liu, C. (2017). Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs. Educational Technology Research and Development, 65, 1195–1214.

    Article  Google Scholar 

  • Yilmaz, A. B., Ünal, M., & Çakir, H. (2017). Evaluating MOOCs according to instructional design principles. Journal of Learning and Teaching in Digital Age, 2(2), 26–35.

    Google Scholar 

  • Yoila, A. O., & Chukwuemeka, E. J. (2015). Instructional design quality evaluation of eastern mediterranean university open courses. International Journal of Scientific Research in Science, Engineering and Technology, 1(6), 1–7.

    Google Scholar 

  • Zarb, M., & Hughes, J. (2015). Breaking the communication barrier: Guidelines to aid communication within pair programming. Computer Science Education, 25, 120–151. https://doi.org/10.1080/08993408.2015.1033125.

    Article  Google Scholar 

  • Zhang, Q., Peck, K. L., Hristova, A., Jablokow, K. W., Hoffman, V., Park, E., et al. (2016). Exploring the communication preferences of MOOC learners and the value of preference-based groups: Is grouping enough? Educational Technology Research and Development, 64, 809–837. https://doi.org/10.1007/s12528-017-9135-3.

    Article  Google Scholar 

  • Zhenghao, C., Alcorn, B., Christensen, G., Eriksson, N., Koller, D., & Emanuel, E. J. (2015). Who’s benefiting from MOOCs, and why. Harvard Business Review. Retrieved January 8, 2018 from https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why.

  • Zhu, M., Bonk, C. J., & Sari, A. R. (2018a). Instructor experiences designing MOOCs in higher education: Pedagogical, resource, and logistical considerations and challenges. Online Learning, 22(4), 203–241. https://doi.org/10.24059/olj.v22i4.1495.

    Article  Google Scholar 

  • Zhu, M., Sari, A., & Lee, M. M. (2018b). A systematic review of research methods and topics of the empirical MOOC literature (2014-2016). The Internet and Higher Education, 37, 31–39.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Education Policy, Organization and Leadership, College of Education, University of Illinois at Urbana-Champaign, 1310 S. 6th St., Champaign, IL, 61820, USA

    Eunjung Grace Oh

  2. Department of Learning and Instruction, Graduate School of Education, University at Buffalo, The State University of New York, 578 Baldy, Buffalo, NY, 14260, USA

    Yunjeong Chang

  3. Faculty of Education, The University of Hong Kong, Pokfulam Rd., Hong Kong, Hong Kong SAR

    Seung Won Park

Authors
  1. Eunjung Grace Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunjeong Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seung Won Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eunjung Grace Oh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oh, E.G., Chang, Y. & Park, S.W. Design review of MOOCs: application of e-learning design principles. J Comput High Educ 32, 455–475 (2020). https://doi.org/10.1007/s12528-019-09243-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12528-019-09243-w

Keywords

Search

Navigation