Abstract
We present lessons learned from an ongoing attempt to conceptualize, develop, and refine a way for teachers to gather formative assessment evidence about classroom argumentation as it happens. The system—named DiALoG (Diagnosing Argumentation Levels of Groups)—includes a digital scoring tool that allows teachers to assess oral classroom argumentation across two primary dimensions: one to capture the Intrapersonal, discipline-specific features of scientific arguments, and another to capture the Interpersonal, group regulatory features of argumentation as a dynamic social act. Coupled with the digital assessment are responsive mini-lessons (RMLs), which provide follow-up curriculum for teachers to respond to different levels of classroom argumentation proficiency for each item assessed. We use classroom observations, interviews, and surveys from piloting science teachers in two different states to iteratively refine this multifaceted formative assessment system of oral classroom argumentation. Lessons learned include the realization by pilot teachers that using the DiALoG system fine-tunes their professional vision to notice student practices they had not previously considered, and the accompanying RMLs help fill gaps in their pedagogical content knowledge and repertoire. Furthermore, while the DiALoG system is intended to be a formative assessment, we learned that the mere presence of numerical scores can queue teacher schema for summative assessment. This prompted us to do away with numbers entirely in the latest version of our digital scoring tool. Such lessons learned from teacher experiences with the development of a novel formative assessment system like DiALoG can be instructive to the development of science educational technology more broadly.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berland, L. K., & McNeill, K. L. (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94(5), 765–793.
Burkhardt, H. (1987). “What You Test Is What You Get.” The Dynamics of Curriculum Change in Developments in School Mathematics Worldwide. Chicago: University of Chicago School Mathematics Project.
Butler, R. (1988). Enhancing and undermining intrinsic motivation: The effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 58(1), 1–14.
Chi, M. T. H. (2009). Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105. https://doi.org/10.1111/j.1756-8765.2008.01005.x
Chi, M., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243.
Common Core State Standards Initiative. (2011). Common core state standards for english language arts & literacy in history/social studies, science, and technical subjects. Retrieved from http://www.corestandards.org/assets/CCSSI_ELA%20Standards.pdf
Darling-Hammond, L., & Wise, A. E. (1985). Beyond standardization: State standards and school improvement. The Elementary School Journal, 85(3), 315–336.
Desoete, A. (2008). Multi-method assessment of metacognitive skills in elementary school children: How you test is what you get. Metacognition and Learning, 3(3), 189.
Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84(3), 287–312.
Erduran, S., Simon, S., & Osborne, J. (2004). TAPping into argumentation: Developments in the application of Toulmin’s Argument Pattern for studying science discourse. Science Education, 88(6), 915–933.
Furtak, E. M., Hardy, I., Beinbrech, C., Shavelson, R. J., & Shemwell, J. T. (2010). A framework for analyzing evidence-based reasoning in science classroom discourse. Educational Assessment, 15(3–4), 175–196.
Goodwin, C. (1994). Professional vision. American Anthropologist, 96(3), 606–633.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.
Kuhn, D., & Udell, W. (2003). The development of argument skills. Child Development, 74(5), 1245–1260.
Kuhn, D., Wang, Y., & Li, H. (2010). Why argue? Developing understanding of the purposes and values of argumentive discourse. Discourse Processes, 48(1), 26–49.
McNeill, K. L., González-Howard, M., Katsh-Singer, R., Price, J. F., & Loper, S. (2013). Teachers’ beliefs and practices around argumentation during a curriculum enactment. Presented at the Annual Meeting of the National Association for Research in Science Teaching (NARST), Puerto Rico.
Michaels, S., O’Connor, C., & Resnick, L. B. (2008). Deliberative discourse idealized and realized: Accountable talk in the classroom and in civic life. Studies in Philosophy and Education, 27(4), 283–297.
Nasir, N. I. S., Rosebery, A. S., Warren, B., & Lee, C. D. (2006). Learning as a cultural process: Achieving equity through diversity. In K. Sawyer (Ed.), Handbook for the learning sciences. (pp. 489–504). Cambridge University Press.
National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press.
NGSS Lead States. (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press. Retrieved May 5, 2021, from http://www.nextgenscience.org/
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218.
Osborne, J. (2010). Arguing to learn in science: The role of collaborative, critical discourse. Science, 328(5977), 463–466.
Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41(10), 994–1020. https://doi.org/10.1002/tea.20035
Pearson, P. D., Moje, E., & Greenleaf, C. (2010). Literacy and science: Each in the service of the other. Science, 328(5977), 459–463.
Sampson, V., Enderle, P. J., & Walker, J. P. (2012). The development and validation of the assessment of scientific argumentation in the classroom (ASAC) observation protocol: A tool for evaluating how students participate in scientific argumentation. In Perspectives on scientific argumentation (pp. 235–264). Springer.
Sherin, M., & Van Es, E. A. (2009). Effects of video club participation on teachers’ professional vision. Journal of Teacher Education, 60(1), 20–37.
Wiliam, D. (2006). Formative assessment: Getting the focus right. Educational Assessment, 11(3–4), 283–289.
Acknowledgements
We would be remiss without acknowledging the amazing efforts of the many hard-working and dedicated teachers that have shared keen insights with our research team. This research has been made possible through the generous support of the following grants: Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum (National Science Foundation, #1621496 and #1621441); Constructing and Critiquing Arguments in Middle School Science Classrooms: Supporting Teachers with Multimedia Educative Curriculum Materials (National Science Foundation, #1119584); and Constructing and Critiquing Arguments: Diagnostic Assessment for Information and Action System (Carnegie Corporation of New York, #B-8780).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The research reported in this manuscript was conducted without any potential conflicts of interest. Human subjects approval was granted, which included an informed consent process for our study participants.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Henderson, J.B., Zillmer, N., Holton, A. et al. How Science Teachers DiALoG Classrooms: Towards a Practical and Responsive Formative Assessment of Oral Argumentation. J Sci Educ Technol 30, 803–815 (2021). https://doi.org/10.1007/s10956-021-09921-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10956-021-09921-4