Abstract
Over three years, our research team has designed various learning supports for promoting content knowledge and solving game levels. In this case study, we examined the optimal design and the evaluation of learning support videos for a physics educational game. Often studies focus on investigating the effects of research-based principles without a systematic examination of the design and development processes. Thus, comprehensive design descriptions and recommendations for developing effective in-game learning supports are scarce in the literature. This study comprises two stages: design and evaluation. In the design stage, we collaborated with two physics experts to design and iteratively revise 18 learning support videos. We applied the First Principles of Instruction (Merrill, 2002) to create instructional strategies and multimedia learning principles (Mayer, 2017) to develop the videos and help learners engage in cognitive processing. In the evaluation stage, we presented the videos to 14 students to gather feedback on their perceptions and, in the following year, examined the effectiveness of the final videos with 263 students. Results revealed that, among all supports, the videos were the only support that significantly predicted posttest scores and game levels completed and viewing patterns did not affect game enjoyment. We conclude with a discussion of our experiences and recommendations to contribute to the foundation of designing in-game learning supports.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alpizar, D., Adesope, O. O., & Wong, R. M. (2020). A meta-analysis of signaling principle in multimedia learning environments. Educational Technology Research and Development, 68, 2095–2119. https://doi.org/10.1007/s11423-020-09748-7
Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233–246. https://doi.org/10.1111/j.2044-8279.1992.tb01017.x
Chiu, T. K. F., & Churchill, D. (2015). Exploring the characteristics of an optimal design of digital materials for concept learning in mathematics: Multimedia learning and variation theory. Computers & Education, 82, 280–291
Chiu, T. K. F., & Churchill, D. (2016). Design of learning objects for concept learning: Effects of multimedia learning principles and an instructional approach. Interactive Learning Environments, 24(6), 1355–1370
Churchill, D. (2013). Conceptual model design and learning uses. Interactive Learning Environments, 21(1), 54–67. https://doi.org/10.1080/10494820.2010.547203
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of educational research, 86(1), 79–122. https://doi.org/10.3102/0034654315582065
Craig, S. D., & Schroeder, N. (2017). Reconsidering the voice effect when learning from a virtual human. Computers & Education, 114, 193–205. https://doi.org/10.1016/j.compedu.2017.07.003
Davis, R., Vincent, J., & Park, T. (2019). Reconsidering the voice principle with non-native language speakers. Computers & Education, 140, 103605. https://doi.org/10.1016/j.compedu.2019.103605
Delacruz, G. C. (2010). Games as formative assessment environments: Examining the impact of explanations of scoring and incentives on math learning, game performance, and help seeking (Publication No. 3446784) [Doctoral dissertation, University of California, Los Angeles]. ProQuest Dissertation Publishing.
Dousay, T. A. (2016). Effects of redundancy and modality on the situational interest of adult learners in multimedia learning. Educational Technology Research and Development, 64, 1251–1271. https://doi.org/10.1007/s11423-016-9456-3
Farrell, S. (2015, October 11). Which Comes First? Layout or Content? Nielsen Norman Group. https://www.nngroup.com/articles/layout-vs-content/.
Gardner, J., Barclay, M., Kong, Y., & LeVally, C. (2020). Designing an accelerated graduate evaluation course using the first principles of instruction and interactive media. Journal of Educational Technology Systems. https://doi.org/10.1177/0047239519893049
Glaser, M., & Schwan, S. (2015). Explaining pictures: How verbal cues influence processing of pictorial learning material. Journal of Educational Psychology, 107(4), 1006–1018. https://doi.org/10.1037/edu0000044
Harskamp, E., Mayer, R. E., & Suhre, C. (2007). Does the modality principle for multimedia learning apply to science classrooms? Learning and Instruction, 17, 465–477. https://doi.org/10.1016/j.learninstruc.2007.09.010
Johnson, C. I., & Mayer, R. (2012). An eye movement analysis of the spatial contiguity effect in multimedia learning. Journal of Experimental Psychology, 18(2), 178–191. https://doi.org/10.1037/a0026923
Klein, J. D., & Mendenhall, A. (2018). Applying the First Principles of Instruction in a short-term, high volume, rapid production of online professional development modules. Journal of Computing in Higher Education, 30, 93–110. https://doi.org/10.1007/s12528-017-9166-9
Ke, F. (2016). Designing and integrating purposeful learning in game play: A systematic review. Educational Technology Research and Development, 64(2), 219–244. https://doi.org/10.1007/s11423-015-9418-1
Lauer, D. A., & Pentak, S. (2011). Design basics. (8th ed.). Wadsworth.
Lo, C. K., & Hew, K. F. (2017). Using “First principles of instruction” to design secondary school mathematics flipped classroom: The findings of two exploratory studies. Educational Technology & Society, 20(1), 222–236
Lo, C. K., Lie, C. W., & H, K.F. . (2018). Applying “First Principles of Instruction” as a design theory of the flipped classroom: Findings from a collective study of four secondary school subjects. Computers & Education, 118, 150–165
Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2019). Role of subjective and objective measures of cognitive processing during learning in explaining the spatial contiguity effect. Learning and Instruction, 61, 23–34. https://doi.org/10.1016/j.learninstruc.2018.12.001
Mayer, R. E. (2017). Using multimedia for e-learning. Journal of Computer Assisted Learning, 33(5), 403–423. https://doi.org/10.1111/jcal.12197
Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59
Mohd, C. K. N. C. K., & Shahbodin, F. (2015). Personalized learning environment: Alpha testing, beta testing & user acceptance test. Procedia - Social and Behavioral Sciences, 195, 837–843. https://doi.org/10.1016/j.sbspro.2015.06.319
Moreno, R., & Mayer, R. E. (2002). Learning science in virtual reality multimedia environments: Role of methods and media. Journal of Educational Psychology, 94, 598–610
Moreno, R., & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19, 309–326
Moreno, R., & Ortegano-Layne, L. (2008). Do classroom exemplars promote the application of principles in teacher education? A comparison of videos, animations, and narratives. Educational Technology Research and Development, 56, 449–465. https://doi.org/10.1007/s11423-006-9027-0
Nielsen, J. (2014, September 1). Demonstrate thinking aloud by showing users a video. Nielsen Norman Group. https://www.nngroup.com/articles/thinking-aloud-demo-video/.
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivation: Classic definitions and new directions. Contemporary Educational Psychology, 25, 54–67
Richey, R. C., & Klein, J. D. (2007). Design and development research. Routledge.
Schrader, C., & Bastiaens, T. (2012). Learning in educational computer games for novices: The impact of support provision types on virtual presence, cognitive load, and learning outcomes. International Review of Research in Open & Distance Learning, 13(3), 206–227
Schwan, S., Lutz, S., & Dreier, F. (2018). Multimedia in the wild: Testing the validity of multimedia learning principles in an art exhibition. Learning and Instruction, 55, 148–157. https://doi.org/10.1016/j.learninstruc.2017.10.004
Shute, V. J., Almond, R. G., & Rahimi, S. (2019). Physics Playground (version 1.3) [computer software]. Tallahassee, FL: Retrieved from https://pluto.coe.fsu.edu/ppteam/pp-links/.
Shute, V. J., Ke, F., Almond, R. G., Rahimi, S., Smith, G., & Lu, X. (2019). How to increase learning while not decreasing the fun in educational games. In R. Feldman (Ed.), Learning Science: Theory, Research, and Practice. (pp. 327–357). McGraw Hill.
Shute, V. J., Rahimi S., Smith, G., Ke, F., Almond, R., Dai, C-P, Kuba, R., Liu, Z., Yang, X., & Sun, C. (2020). Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity, and learning supports in educational games. Journal of Computer-Assisted Learning,37(1), 127–141. https://doi.org/10.1111/jcal.12473
Shute, V. J., Smith, G., Kuba, R., Dai, C-P., Rahimi, S., Liu, Z., & Almond, R. G. (2020). The design, development, and testing of learning supports for the Physics Playground game. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-020-00196-1
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer.
Sweller, J. (2020). Cognitive load theory and educational technology. Educational Technology Research and Development, 68, 1–16. https://doi.org/10.1007/s11423-019-09701-3
Tu, W., & Snyder, M. M. (2017). Developing conceptual understanding in a statistics course: Merrill’s First Principles and real data at work. Educational Technology Research and Development, 65, 579–595. https://doi.org/10.1007/s11423-016-9482-1
Van Eck, R., & Dempsey, J. (2002). The effect of competition and contextualized advisement on the transfer of mathematics skills in a computer-based instructional simulation game. Educational Technology Research and Development, 50(3), 23–41. https://doi.org/10.1007/BF02505023
Wouters, P., & van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, 60(1), 412–425. https://doi.org/10.1016/j.compedu.2012.07.018
Yang, L., Li, J., Lu, W., Chen, Y., Zhang, K., & Li, Y. (2020). The influence of font scale on semantic expression of word cloud. Journal of Visualization, 23, 981–998. https://doi.org/10.1007/s12650-020-00678-3
Acknowledgments
This work was supported by the National Science Foundation, United States [award number #037988] and the Department of Education, United States [award number #039019]. We want to acknowledge Russell Almond, Fengfeng Ke, Curt Fulwider, Zhichun Liu, Chen Sun, and Jiawei Li for helping in different stages of this project.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kuba, R., Rahimi, S., Smith, G. et al. Using the first principles of instruction and multimedia learning principles to design and develop in-game learning support videos. Education Tech Research Dev 69, 1201–1220 (2021). https://doi.org/10.1007/s11423-021-09994-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-021-09994-3