Abstract
Kinematics is an important but challenging area in physics. In previously published works of the current research project, it was revealed that there is a significant relationship between facial microexpression states (FMES) changes and conceptual conflict-induced conceptual change. Consequently, the current study integrated FMES into a kinematics multiple representation instructional scenario to investigate if FMES could be used to help construct students’ conceptual paths, and help predict students’ learning outcome. Analysis revealed that types of students’ FMES (neutral, surprised, positive, and negative) were important in helping instructors predict students’ learning outcomes. Findings showed that exhibiting negative FMES through all three major representation segments of the instructional process (i.e., scientific demonstration, textual instruction, and animated instruction) suggests a higher probability of conceptual change among students with sufficient background knowledge on the topic. For students with insufficient prior knowledge, the result was the opposite. Moreover, animated representation was found to be critical to the prediction of student conceptual change. In sum, the results showed FMES as a viable indicator for conceptual change in kinematics, and also reaffirmed the importance of prior knowledge and representations of scientific concepts.
Similar content being viewed by others
Availability of Data and Material
The data and relevant material may be accessed with authorization from the author(s) or National Taiwan Normal University.
References
Ainsworth, S. (1999). The functions of multiple representations. Computers in Education, 33(2), 131–152. https://doi.org/10.1016/S0360-1315(99)00029-9.
Ainsworth, S. (2008). How should we evaluate multimedia learning environments? In J.-F. Rouet, R. Lowe, & W. Schnotz (Eds.), Understanding multimedia documents (pp. 249–265). Boston: Springer US.
Ainsworth, S. (2014). The multiple representation principle in multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 464–486). Cambridge: Cambridge University Press.
Anderson, J. L., & Wall, S. D. (2016). Kinecting physics: conceptualization of motion through visualization and embodiment. Journal of Science Education and Technology, 25(2), 161–173. https://doi.org/10.1007/s10956-015-9582-4.
Barrett, T. J., Stull, A. T., Hsu, T. M., & Hegarty, M. (2015). Constrained interactivity for relating multiple representations in science: when virtual is better than real. Computers in Education, 81, 69–81.
Bellocchi, A., & Ritchie, S. M. (2015). “I was proud of myself that I didn't give up and I did it”: experiences of pride and triumph in learning science. Science Education, 99(4), 638–668. https://doi.org/10.1002/sce.21159.
Bellocchi, A., Ritchie, S. M., Tobin, K., King, D., Sandhu, M., & Henderson, S. (2014). Emotional climate and high quality learning experiences in science teacher education. Journal of Research in Science Teaching, 51(10), 1301–1325. https://doi.org/10.1002/tea.21170.
Braasch, J. L. G., & Goldman, S. R. (2010). The role of prior knowledge in learning from analogies in science texts. Discourse Processes, 47(6), 447–479. https://doi.org/10.1080/01638530903420960.
Breiman, L., Friedman, J. H., Stone, C. J., & Olshen, R. A. (1984). Classification and regression trees. Pacific Grove: Wadsworth.
Chiu, M.-H., Chou, C.-C., Wu, W.-L., & Liaw, H. (2014). The role of facial microexpression state (FMES) change in the process of conceptual conflict. British Journal of Educational Technology, 45(3), 471–486. https://doi.org/10.1111/bjet.12126.
Chiu, M.-H., Liaw, H. L., Yu, Y.-R., & Chou, C.-C. (2019). Facial micro-expression states as an indicator for conceptual change in students' understanding of air pressure and boiling points. British Journal of Educational Technology, 50(1), 469–480. https://doi.org/10.1111/bjet.12597.
Chiu, M.-H., Yu, Y.-R., Liaw, H. L., & Lin, C.-H. (2015). The use of facial micro-expression state and Tree-Forest Model for predicting conceptual-conflict based conceptual change. Helsinki: Paper presented at the European Science Education Research Association Conference.
Cook, M., Wiebe, E. N., & Carter, G. (2008). The influence of prior knowledge on viewing and interpreting graphics with macroscopic and molecular representations. Science Education, 92(5), 848–867.
De Ambrosis, A., Malgieri, M., Mascheretti, P., & Onorato, P. (2015). Investigating the role of sliding friction in rolling motion: a teaching sequence based on experiments and simulations. European Journal of Physics, 36(3), 21. https://doi.org/10.1088/0143-0807/36/3/035020.
Ekman, P. (1970). Universal facial expressions of emotion. California Mental Health Research Digest, 8, 151–158.
Fang, N., & Uziak, J. (2018). Student misconceptions of general plane motion in rigid-body kinematics. Journal of Professional Issues in Engineering Education and Practice, 144(3), 03118001. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000366.
Flores, R., Koontz, E., Inan, F. A., & Alagic, M. (2015). Multiple representation instruction first versus traditional algorithmic instruction first: Impact in middle school mathematics classrooms. Educational Studies in Mathematics, 89(2), 267–281.
Fredricks, J. A., & McColskey, W. (2012). The measurement of student engagement: a comparative analysis of various methods and student self-report instruments. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 763–782). Boston: Springer US.
Fyfe, E. R., Rittle-Johnson, B., & DeCaro, M. S. (2012). The effects of feedback during exploratory mathematics problem solving: prior knowledge matters. Journal of Education & Psychology, 104(4), 1094–1108. https://doi.org/10.1037/a0028389.
Gasper, K. (2003). When necessity is the mother of invention: mood and problem solving. Journal of Experimental Social Psychology, 39(3), 248–262 Retrieved from http://www.sciencedirect.com/science/article/pii/S0022103103000234. Accessed 10 Sept 2020.
Gilbert, J. K., & Treagust, D. F. (2009). Introduction: macro, submicro and symbolic representations and the relationship between them: Key models in chemical education. In Gilbert, J. K. & Treagust, D. F. (Eds.), Multiple representations in chemical education (pp. 1-8): Springer.
Grayson, D. J., & McDermott, L. C. (1996). Use of the computer for research on student thinking in physics. American Journal of Physics, 64(5), 557–565.
Hailikari, T., Katajavuori, N., & Lindblom-Ylanne, S. (2008). The relevance of prior knowledge in learning and instructional design. American Journal of Pharmaceutical Education, 72(5), 113. https://doi.org/10.5688/aj7205113.
Hewitt, P. (1999). Average speed of balls. Physics Teacher, 37(7), 423–423.
Hwang, G.-J., Chu, H.-C., Shih, J.-L., Huang, S.-H., & Tsai, C.-C. (2010). A decision-tree-oriented guidance mechanism for conducting nature science observation activities in a context-aware ubiquitous learning environment. Journal of Educational Technology & Society, 13(2), 53–64 Retrieved from http://www.jstor.org/stable/jeductechsoci.13.2.53.
Kalyuga, S. (2008). Relative effectiveness of animated and static diagrams: an effect of learner prior knowledge. Computers in Human Behavior, 24(3), 852–861. https://doi.org/10.1016/j.chb.2007.02.018.
Kenekayoro, P., Buckley, K., & Thelwall, M. (2014). Automatic classification of academic web page types. Scientometrics, 101(2), 1015–1026.
Kennedy, G., Coffrin, C., Barba, P. d., & Corrin, L. (2015). Predicting success: how learners’ prior knowledge, skills and activities predict MOOC performance. Poughkeepsie: Paper presented at the Proceedings of the Fifth International Conference on Learning Analytics and Knowledge.
Kim, N.-G., & Son, H. (2015). How facial expressions of emotion affect distance perception. Frontiers in Psychology, 6(1825). https://doi.org/10.3389/fpsyg.2015.01825.
King, D., Ritchie, S., Sandhu, M., & Henderson, S. (2015). Emotionally intense science activities. International Journal of Science Education, 37(12), 1886–1914. https://doi.org/10.1080/09500693.2015.1055850.
Kirby, N. F., & Dempster, E. R. (2014). Using decision tree analysis to understand foundation science student performance. Insight gained at one South African university. International Journal of Science Education, 36(17), 2825–2847.
Kozma, R. B. (2000). The use of multiple representations and the social construction of understanding in chemistry. In Jacobson, M. & Kozma, R. (Eds.), Innovations in science and mathematics education: advanced designs for technologies of learning (pp. 11-46): Routledge.
Kozma, R. B., & Russell, J. (1997). Multimedia and understanding: expert and novice responses to different representations of chemical phenomena. Journal of Research in Science Teaching, 34(9), 949–968 Retrieved from https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291098-2736%28199711%2934%3A9%3C949%3A%3AAID-TEA7%3E3.0.CO%3B2-U.
Leonard, W. J., & Gerace, W. J. (1996). The power of simple reasoning. Physics Teacher, 34(5), 280–283.
Lewinski, P., den Uyl, T. M., & Butler, C. (2014). Automated facial coding: Validation of basic emotions and FACS AUs in FaceReader. Journal of Neuroscience, Psychology, and Economics, 7(4), 227–236. https://doi.org/10.1037/npe0000028.supp.(Supplemental).
Liaw, H. L., Chiu, M.-H., & Chou, C.-C. (2014). Using facial recognition technology in the exploration of student responses to conceptual conflict phenomenon. Chemistry Education Research and Practice, 15(4), 824–834. https://doi.org/10.1039/c4rp00103f.
Limón, M., & Carretero, M. (1997). Conceptual change and anomalous data: a case study in the domain of natural sciences. European Journal of Psychology of Education, 12(2), 213–230. https://doi.org/10.1007/bf03173085.
Linenberger, K. J., & Bretz, S. L. (2012). Generating cognitive dissonance in student interviews through multiple representations. Chemistry Education Research and Practice, 13(3), 172–178.
Loyens, S. M. M., Jones, S. H., Mikkers, J., & van Gog, T. (2015). Problem-based learning as a facilitator of conceptual change. Learning and Instruction, 38, 34–42. https://doi.org/10.1016/j.learninstruc.2015.03.002.
Madu, B. C., & Orji, E. (2015). Effects of cognitive conflict instructional strategy on students’ conceptual change in temperature and heat. SAGE Open, 5(3), 215824401559466. https://doi.org/10.1177/2158244015594662.
Maison, D., & Pawłowska, B. (2017). Using the Facereader method to detect emotional reaction to controversial advertising referring to sexuality and homosexuality. In Neuroeconomic and Behavioral Aspects of Decision Making (pp. 309-327): Springer.
Matsumoto, D., & Hwang, H. (2011). Evidence for training the ability to read microexpressions of emotion. Motivation and Emotion, 35(2), 181–191. https://doi.org/10.1007/s11031-011-9212-2.
Mayer, R. E. (2003). The promise of multimedia learning: using the same instructional design methods across different media. Learning and Instruction, 13(2), 125–139. https://doi.org/10.1016/S0959-4752(02)00016-6.
Mayer, R. E., & Moreno, R. (2005). A cognitive theory of multimedia learning: implications for design principles 91.
McNamara, D. S., & Kintsch, W. (1996). Learning from texts: effects of prior knowledge and text coherence. Discourse Processes, 22(3), 247–288.
Moore, J. C. (2009). Two tracks demonstrate average speed. Physics Education, 44(5), 456–458.
Ninaus, M., Greipl, S., Kiili, K., Lindstedt, A., Huber, S., Klein, E., Karnath, H. O., & Moeller, K. (2019). Increased emotional engagement in game-based learning—a machine learning approach on facial emotion detection data. Computers in Education, 142, 103641. https://doi.org/10.1016/j.compedu.2019.103641.
Ozuru, Y., Dempsey, K., & McNamara, D. S. (2009). Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learning and Instruction, 19(3), 228–242. https://doi.org/10.1016/j.learninstruc.2008.04.003.
Pekrun, R. (2019). Inquiry on emotions in higher education: progress and open problems. Studies in Higher Education, 44(10), 1806–1811. https://doi.org/10.1080/03075079.2019.1665335.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105. https://doi.org/10.1207/s15326985ep3702_4.
Pekrun, R., Lichtenfeld, S., Marsh, H. W., Murayama, K., & Goetz, T. (2017). Achievement emotions and academic performance: longitudinal models of reciprocal effects. Child Development, 88(5), 1653–1670. https://doi.org/10.1111/cdev.12704.
Rau, M. A. (2018). Making connections among multiple visual representations: how do sense-making skills and perceptual fluency relate to learning of chemistry knowledge? Instructional Science, 46(2), 209–243. https://doi.org/10.1007/s11251-017-9431-3.
Rimoldini, L. G., & Singh, C. (2005). Student understanding of rotational and rolling motion concepts. Physical Review Special Topics - Physics Education Research, 1(1), 010102.
Skiendziel, T., Rösch, A. G., & Schultheiss, O. C. (2019). Assessing the convergent validity between the automated emotion recognition software Noldus FaceReader 7 and Facial Action Coding System Scoring. PLoS One, 14(10), 1–18. https://doi.org/10.1371/journal.pone.0223905.
Staus, N. L., & Falk, J. H. (2017). The role of emotion in informal science learning: testing an exploratory model. Mind, Brain, and Education, 11(2), 45–53. https://doi.org/10.1111/mbe.12139.
Trevors, G., Duffy, M., & Azevedo, R. (2014). Note-taking within MetaTutor: interactions between an intelligent tutoring system and prior knowledge on note-taking and learning. Educational Technology Research and Development, 62(5), 507–528. https://doi.org/10.1007/s11423-014-9343-8.
Tyng, C. M., Amin, H. U., Saad, M. N. M., & Malik, A. S. (2017). The influences of emotion on learning and memory. Frontiers in Psychology, 8(1454). https://doi.org/10.3389/fpsyg.2017.01454.
Ubben, I., Salisbury, S. L., & Daniel, K. L. (2019). Combining visual and verbal data to diagnose and assess modeling competence. In A. Upmeier zu Belzen, D. Krüger, & J. van Driel (Eds.), Towards a competence-based view on models and modeling in science education (pp. 99–115). Cham: Springer International Publishing.
van Heuvelen, A., & Zou, X. (2001). Multiple representations of work-energy processes. American Journal of Physics, 69(2), 184–196.
Wang, J., Berzins, K., Hicks, D., Malkers, J., Xiao, F., & Pinheiro, D. (2012). A boosted-tree method for name disambiguation. Scientometrics, 93(2), 391–411.
White, R., & Gunstone, R. (1992). Prediction-observation-explanation. In R. White & R. Gunstone (Eds.), Probing Understanding (pp. 44–64). London: Falmer Press.
Wu, C.-H., Huang, Y.-M., & Hwang, J.-P. (2016). Review of affective computing in education/learning: trends and challenges. British Journal of Educational Technology, 47(6), 1304–1323. https://doi.org/10.1111/bjet.12324.
Xie, Z., Yu, X., Niu, J., & Li, Y. (2019). Facial microexpression recognition based on adaptive key frame representation. Journal of Electronic Imaging, 28(3), 033015.
Zhai, X. (2019). Applying machine learning in science assessment: opportunity and challenge. Journal of Science Education and Technology 1-4.
Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L. (2020). Applying machine learning in science assessment: a systematic review. Studies in Science Education, 56(1), 111–151.
Acknowledgments
This project was supported by grants from the Ministry of Science and Technology in Taiwan (MOST104-2511-S-003-047-MY2 and 106-2511-S-003-009-MY2; Post-Doc 102-2811-S-003-006 and Post-Doc 102-2811-S-003-007). The authors would like to express their appreciation for to the Ministry funding this project and to the students who participated in this research.
Funding
This project was supported by grants from the Taiwanese Ministry of Science and Technology (MOST104-2511-S-003-047-MY2 and 106-2511-S-003-009-MY2; Post-Doc 102-2811-S-003-006 and Post-Doc 102-2811-S-003-007).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The current study had been reviewed and approved by the Internal Review Board of the authors’ university.
Conflict of Interest
The authors declare that they have no conflict of interest.
Code Availability
The code may be accessed with authorization from the author(s) or National Taiwan Normal University.
Informed Consent
Informed consent was obtained from all participants.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Sample Questions from the Screening Test and the Post-Test
(All questions translated from Traditional Chinese)
Screening Test:
What would you see when a speeding car runs off the cliff?
-
1
It will fall perpendicularly.
-
2
It will fall diagonally.
-
3
It will fall in a curve.
-
4
It will not fall.
Post-Test:
In the figure below, how would the horizontal velocity of the metal ball change just before it leaves the horizontal surface?
-
1
Increase
-
2
Decrease
-
3
Remain Unchanged
In the figure below, how would the horizontal velocity of the metal ball change just before it slides down the slope?
-
1
Increase
-
2
Decrease
-
3
Remain Unchanged
Rights and permissions
About this article
Cite this article
Liaw, H., Yu, YR., Chou, CC. et al. Relationships between Facial Expressions, Prior Knowledge, and Multiple Representations: a Case of Conceptual Change for Kinematics Instruction. J Sci Educ Technol 30, 227–238 (2021). https://doi.org/10.1007/s10956-020-09863-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10956-020-09863-3