Tracking the process of data use professional development interventions for instructional improvement: A systematic literature review☆

https://doi.org/10.1016/j.edurev.2020.100362Get rights and content

Highlights

  • •

    The present study provides evidence on the effects of data use professional development interventions (PDIs) on teacher and student outcomes.

  • •

    A framework is proposed for evaluating data use PDI pathways from the interventions to the practices of data use and finally the outcomes.

  • •

    The effects on student outcomes are not straightforward but mediated by teacher outcomes, practices of data use and instructional changes.

  • •

    There is no evidence in the studies that PD pathways were, respectively, evaluated to support claims on their effects on student outcomes.

Abstract

This paper investigates the evidence on the effects of data use professional development interventions (PDIs) on teacher and student outcomes through a systematic literature review. More specifically, it focuses on the key features of data use PDIs, the practices of data use, and its effects on teacher and student outcomes. To that end, we propose a framework for evaluating data use PDIs that moves from the interventions themselves (input) to the practices of data use (process) and finally the outcomes (output). The framework is used as the theoretical underpinning of this review. The results provide insights into the importance of incorporating some or all key PD features. The process of data use is viewed in this paper not only as an improvement strategy with a three-part interrelated and cyclical process of intentionally using data to inform instruction but as a constructivist learning process as well. Data use PDIs have promising results on teacher satisfaction, data literacy, attitudes and beliefs, as well as student subject-related outcomes. However, the findings also reveal that the effects on student outcomes are mediated by teacher outcomes, practices of data use and instructional changes. Overall, despite those findings, there is no evidence in the studies that PD features, teacher outcomes, practices of data use and instructional changes were, respectively, evaluated to support claims on their effects on student outcomes.

Section snippets

Data use professional development pathways for instructional improvement

Understanding the linkages of data use PDIs that move from interventions themselves to data use practices and finally outcomes is important as it can provide a foundation for understanding the pathways by which an intervention - like a data use PDI for teachers - produces outcomes (Rogge, Agasisti, & De Witte, 2017; Vanthienen & De Witte, 2017). Desimone (2009) proposed a pathway model that explains how student outcomes improve when teachers participate in PD. This model shows interactive

Method

This review went through the phases recommended by Petticrew and Roberts’s (2006) method for systematic reviews in social sciences. The main question is what is the evidence of the effects of comprehensive data use professional development interventions (PDIs) on teacher and student outcomes? This question has three sub-questions:

  • 1)

    What features guide the design of the data use PDIs?

  • 2)

    What data use processes do the teachers follow during the PDIs?

  • 3)

    What effects do the PDIs have on teacher and student

Results

Based on the framework, this section describes the findings concerning the features of data use PDIs, as well as data use practices and outcomes respectively.

The domain of practice: The process of data use

The results for our research question two concerning the practice of data use processes for instructional improvement show several models that follow systematic (iterative) data use processes for instructional improvement, such as the Four DBDM component model (Staman et al., 2017; van der Scheer et al., 2017; van der Scheer & Visscher, 2016, 2018; van Geel et al., 2016, 2017), the Data Literacy for Teaching (DLfT) (Reeves & Chiang, 2018), the Multi-component Model (van Kuijk et al., 2016), and

The domain of impact: PDI effects

The results of the research question three about the effects of the data use PDIs on teacher and student outcomes are presented below. In general, more than half of the studies investigated the effects of the PDIs on students’ outcomes, whereas the others focussed on teacher outcomes and one study on instructional changes.

Discussion

This systematic literature review contributes to the existing literature by evaluating the components of the PD pathways in the context of data use PDIs. Considering the significance of the underlying processes of data use in understanding its pathways (Coburn & Turner, 2011), the practice of data use was included in our proposed framework that was developed based on the general PD pathways by Desimone (2009) and Merchie et al. (2016). This framework further served as a basis for this review.

Conclusion

The main goal of this review was to investigate the evidence on the effects of data use PDIs on teacher and student outcomes. More specifically, this review contributes to the existing literature by presenting the linkages of the key features of data use PDIs, the practices of data use, and their effects on teacher and student outcomes.

Since data use has gained much attention due to its potential impacts on student outcomes, professional development interventions (PDIs) were developed and

Authors’ statement

The authors certify that they have all contributed to all the research actions (e.g. the concept, design, analysis, writing, or revision of the manuscript). Furthermore, the authors also certify that this manuscript or similar material has not been submitted or published elsewhere.

Acknowledgements

AcknowledgementsThis research was sponsored by the 5000-Doctor Overseas Programme, Ministry of Religious Affairs Republic of Indonesia.

References (64)

  • R.K. Blank et al.

    Does teacher professional development have effects on teaching and learning? Evaluation findings from programs in 14 states

    (2008)
  • S. Borg

    Evaluating the impact of professional development

    RELC Journal

    (2018)
  • D. Carlson et al.

    A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement

    Educational Evaluation and Policy Analysis

    (2011)
  • C.E. Coburn et al.

    Research on data use: A framework and analysis

    Measurement

    (2011)
  • B. Compen et al.

    The impact of teacher engagement in an interactive webinar series on the effectiveness of financial literacy education

    British Journal of Educational Technology

    (in press, 2020)
  • B. Compen et al.

    The role of teacher professional development in financial literacy education: A systematic literature review

    Educational Research Review

    (2019)
  • L. Darling-Hammond et al.

    Effective teacher professional development

    (2017)
  • A. Datnow et al.

    Teacher capacity for and beliefs about data-driven decision making: A literature review of international research

    Journal of Educational Change

    (2016)
  • L. Desimone

    Improving impact studies of teachers' professional development: Toward better conceptualizations and measures

    Educational Researcher

    (2009)
  • L. Desimone et al.

    Best practices in teachers' professional development in the United States

    Psychology, Society, & Education

    (2015)
  • J. Ebbeler et al.

    The effects of a data use intervention on educators' satisfaction and data literacy

    Educational Assessment, Evaluation and Accountability

    (2017)
  • M. van Geel et al.

    Assessing the effects of a school-wide data-based decision-making intervention on student achievement growth in primary schools

    American Educational Research Journal

    (2016)
  • M. van Geel et al.

    Changes in educators' data literacy during a data-based decision making intervention

    Teaching and Teacher Education

    (2017)
  • G. Gelderblom et al.

    Data-based decision making for instructional improvement in primary education

    International Journal of Educational Research

    (2016)
  • T.R. Guskey et al.

    What works in professional development?

    Phi Delta Kappan

    (2009)
  • L. Hamilton et al.

    Using student achievement data to support instructional decision making

    (2009)
  • I. Hoogland et al.

    Prerequisites for data-based decision making in the classroom: Research evidence and practical illustrations

    Teaching and Teacher Education

    (2016)
  • T. Houtveen et al.

    Improving reading achievements of struggling learners

    School Effectiveness and School Improvement

    (2012)
  • G.S. Ikemoto et al.

    Cutting through the “data-driven” mantra: Different conceptions of data-driven decision making

    The Yearbook of the National Society for the Study of Education

    (2007)
  • B. Jeanpierre et al.

    Characteristics of professional development that effect change in secondary science teachers' classroom practices

    Journal of Research in Science Teaching

    (2005)
  • J.B. Jimerson et al.

    Implementing a data use intervention in a United States context: Enabling and constraining factors (2019, April) Paper Presented at the Annual Conference of the

    (2019)
  • C.C. Johnson et al.

    A study of the effect of sustained, whole-school professional development on student achievement in science

    Journal of Research in Science Teaching

    (2007)
  • W.B. Kippers et al.

    Data literacy: What do educators learn and struggle with during a data use intervention?

    Studies In Educational Evaluation

    (2018)
  • S. Konstantopoulos et al.

    The Impact of Indiana's system of interim assessments on Mathematics and Reading achievement

    Educational Evaluation and Policy Analysis

    (2013)
  • M.F. van Kuijk et al.

    Goals, data use, and instruction: The effect of a teacher professional development program on reading achievement

    School Effectiveness and School Improvement

    (2016)
  • M.K. Lai et al.

    The impact of data use professional development on student achievement

    Teaching and Teacher Education

    (2016)
  • M.K. Lai et al.

    Sustained acceleration of achievement in reading comprehension: The New Zealand experience

    Reading Research Quarterly

    (2009)
  • M.K. Lai et al.

    Data-based decision making: An overview

  • M.K. Lai et al.

    Improving achievement in secondary schools: Impact of a literacy project on reading comprehension and secondary school qualifications

    Reading Research Quarterly

    (2014)
  • J.R. Lockwood et al.

    Linking reading coaches and student achievement: Evidence from Florida middle schools

    Educational Evaluation and Policy Analysis

    (2010)
  • E.B. Mandinach

    A perfect time for data use: Using data-driven decision making to inform practice

    Educational Psychologist

    (2012)
  • E.B. Mandinach et al.

    Data-driven decision making: Components of the enculturation of data use in education

    Teachers College Record

    (2015)

    • Cited by (8)

    • Does learning how to use data mean being motivated to use it? Effects of a data use intervention on data literacy and motivational beliefs of pre-service teachers

      2023, Learning and Instruction

    • Teachers’ preferences for online professional development: Evidence from a discrete choice experiment

      2022, Teaching and Teacher Education
      Citation Excerpt :

      Therefore, more research is necessary to examine the effect of certificates on such a type of OTPD. Although studies highlight the importance of a longer duration for in-person instructional data use interventions (Ansyari, Groot, & De Witte, 2020), the teachers prefer a short OTPD duration for instructional data use. As reported by Reeves and Chiang (2018; 2019), an online data use intervention with a short duration can also have effects on teachers' beliefs, self-efficacy, and practices.

    • Teachers’ use of data from digital learning platforms for instructional design: a systematic review

      2024, Educational Technology Research and Development

    • Interpretable early warning recommendations in interactive learning environments: a deep-neural network approach based on learning behavior knowledge graph

      2023, Humanities and Social Sciences Communications

    • E-Learning Pricing Model Policy for Higher Education

      2023, IEEE Access

    • A systematic review and meta-analysis of data use professional development interventions

      2022, Journal of Professional Capital and Community
    View all citing articles on Scopus
    ☆

    This research was sponsored by the 5000-Doctor Overseas Programme, Ministry of Religious Affairs Republic of Indonesia.

    View full text
    © 2020 Elsevier Ltd. All rights reserved.