Abstract
Access to raw data of graphs presented in original articles to calculate the effect size of single-case research is a challenge for researchers conducting studies such as meta-analysis. Researchers typically use data extraction software programs to extract raw data from the graphs in articles. In this study, we aimed to analyze the validity and reliability of the PlotDigitizer software program, which is widely used in literature and an alternative to other data extraction programs, on computers with different operating systems. We performed the digitization of 6.846 data points on three different computers using 15 hypothetical graphs with 20 data series and 186 graphs with 242 data series from 29 published articles to accomplish the goal. Besides, using the values we digitized, we recalculated the 23 effect sizes presented in the original articles for validity analysis. Based on our sampling, we calculated intercoder and intracoder Pearson correlation coefficients. The results showed that PlotDigitizer could be an alternative to other programs as it is free and can run on many current and outdated systems, and it is valid and reliable as it is nearly perfect. Based on the obtained results and considering the data extraction process, we presented various recommendations for the researchers that will use the PlotDigitizer program for the quantitative analysis of single-case graphs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aydin, O., & Diken, I. H. (2020). Studies comparing augmentative and alternative communication systems (AAC) applications for individuals with autism spectrum disorder: A systematic review and meta-analysis. Education & Training in Autism and Developmental Disabilities, 55(2), 119–141
Aydin, O., & Tekin-İftar, E. (2020). Teaching math skills to individuals with autism spectrum disorder: A descriptive and meta-analysis in single case research designs. Ankara University Faculty of Educational Sciences Journal of Special Education, 21(2), 383–426. https://doi.org/10.21565/ozelegitimdergisi.521232.
Arizona-Software. (2008). GraphClick (version 3.0) [computer software]. Arizona-Software. http://www.arizona-software.ch.
Biosoft. (2004). UnGraph for Windows (version 5.0). Author.
Bormann, I. (2012). DigitizeIt (version 2.0). DigitizeIt-Software. http://www.digitizeit.de/.
Boyle, M. A., Samaha, A. L., Rodewald, A. M., & Hoffmann, A. N. (2013). Evaluation of the reliability and validity of GraphClick as a data extraction program. Computers in Human Behavior, 29(3), 1023–1027. https://doi.org/10.1016/j.chb.2012.07.031.
Busse, R. T., McGill, R. J., & Kennedy, K. S. (2015). Methods for assessing single-case school-based intervention outcomes. Contemporary School Psychology, 19, 136–144. https://doi.org/10.1007/s40688-014-0025-7.
Cook, B. G., Lloyd, J. W., Mellor, D., Nosek, B. A., & Therrien, W. J. (2018). Promoting open science to increase the trustworthiness of evidence in special education. Exceptional Children, 85(1), 104–118. https://doi.org/10.1177/0014402918793138.
Drevon, D., Fursa, S. R., & Malcolm, A. L. (2017). Intercoder reliability and validity of WebPlotDigitizer in extracting graphed data. Behavior Modification, 41(2), 323–339. https://doi.org/10.1177/0145445516673998.
Flower, A., McKenna, J. W., & Upreti, G. (2016). Validity and reliability of GraphClick and DataThief III for data extraction. Behavior Modification, 40(3), 396–413. https://doi.org/10.1177/0145445515616105.
Geomatix. (2007). XY digitizer. Author. http://www.geomatix.net/xyit.
Huwaldt, J. A., & Steinhorst, S. (2015). Plot digitizer 2.6.8. PlotDigitizer-Software. http://plotdigitizer.sourceforge.net.
Huwaldt, J. A., & Steinhorst, S. (2020). Plot digitizer 2.6.9. PlotDigitizer-Software. http://plotdigitizer.sourceforge.net
Kadic, A. J., Vucic, K., Dosenovic, S., Sapunar, D., & Puljak, L. (2016). Extracting data from figures with software was faster, with higher interrater reliability than manual extraction. Journal of Clinical Epidemiology, 74, 119–123. https://doi.org/10.1016/j.jclinepi.2016.01.002.
Kennedy, C. H. (2005). Single-case designs for educational research. Pearson/Allyn Bacon.
Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/wwc_scd.pdf
Kratochwill, T. R., Levin, J. R., Horner, R. H., & Swoboda, C. M. (2014). Visual analysis of single-case intervention research: Conceptual and methodological issues. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case intervention research: Methodological and statistical advances (pp. 91–125). American Psychological Association.
Maggin, D. M., Lane, K. L., & Pustejovsky, J. E. (2017). Introduction to the special issue on single-case systematic reviews and meta-analyses. Remedial & Special Education, 38(6), 323–330. https://doi.org/10.1177/0741932517717043.
Maggin, D. M., & Odom, S. L. (2014). Evaluating single-case research data for systematic review: A commentary for the special issue. Journal of School Psychology, 52(2), 237–241. https://doi.org/10.1016/j.jsp.2014.01.002.
Manolov, R., & Rochat, L. (2015). Further developments in summarising and meta-analysing single-case data: An illustration with neurobehavioural interventions in acquired brain injury. Neuropsychological Rehabilitation, 25(5), 637–662. https://doi.org/10.1080/09602011.2015.1064452.
Manolov, R., Solanas, A., & Sierra, V. (2019). Extrapolating baseline trend in single-case data: Problems and tentative solutions. Behavior Research Methods, 51(6), 2847–2869. https://doi.org/10.3758/s13428-018-1165-x.
Manolov, R., & Vannest, K. J. (2019). A visual aid and objective rule encompassing the data features of visual analysis. Behavior Modification. Advance online publication. https://doi.org/10.1177/0145445519854323
Moeyaert, M., Maggin, D., & Verkuilen, J. (2016). Reliability, validity, and usability of data extraction programs for single-case research designs. Behavior Modification, 40(6), 874–900. https://doi.org/10.1177/0145445516645763.
Odom, S. L., Barton, E. E., Reichow, B., Swaminathan, H., & Pustejovsky, J. E. (2018). Between-case standardized effect size analysis of single case designs: Examination of the two methods. Research in Developmental Disabilities, 79, 88–96. https://doi.org/10.1016/j.ridd.2018.05.009.
Olive, M. L., & Franco, J. H. (2008). (Effect) size matters: And so does the calculation. The Behavior Analyst Today, 9(1), 5–10. https://doi.org/10.1037/h0100642.
Parker, R. I., & Vannest, K. J. (2009). An improved effect size for single case research: Nonoverlap of all pairs (NAP). Behavior Therapy, 40(4), 357–367. https://doi.org/10.1016/j.beth.2008.10.006.
Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for single-case research. Exceptional Children, 75(2), 135–150. https://doi.org/10.1177/00144029090750020.
Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42(2), 284–299. https://doi.org/10.1016/j.beth.2010.08.006.
Radley, K. C., Moore, J. W., Dart, E. H., Ford, W. B., & Helbig, K. A. (2019). The effects of lag schedules of reinforcement on social skill accuracy and variability. Focus on Autism & Other Developmental Disabilities, 34(2), 67–80. https://doi.org/10.1177/1088357618811608.
Rakap, S., Rakap, S., Evran, D., & Cig, O. (2016). Comparative evaluation of the reliability and validity of three data extraction programs: UnGraph, GraphClick, and DigitizeIt. Computers in Human Behavior, 55, 159–166. https://doi.org/10.1016/j.chb.2015.09.008.
Riley-Tillman, T. C., & Burns, M. K. (2020). Evaluating educational interventions: Single-case design for measuring response to intervention (2nd ed.). Guilford Press.
Rochat, L., Manolov, R., & Billieux, J. (2018). Efficacy of metacognitive therapy in improving mental health: A meta-analysis of single-case studies. Journal of Clinical Psychology, 74(6), 896–915. https://doi.org/10.1002/jclp.22567.
Rohatgi, A. (2015). WebPlotDigitizer (Version 3.9) [Computer software]. WebPlotDigitizer Online Software. http://arohatgi.info/WebPlotDigitizer
Scruggs, T. E., Mastropieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research methodology and validation. Remedial & Special Education, 8(2), 24–33. https://doi.org/10.1177/074193258700800206.
Shadish, W. R., Brasil, I. C., Illingworth, D. A., White, K. D., Galindo, R., Nagler, E. D., & Rindskopf, D. M. (2009). Using UnGraph to extract data from image files: Verification of reliability and validity. Behavior Research Methods, 41(1), 177–183. https://doi.org/10.3758/BRM.41.1.177.
Shadish, W. R., Hedges, L. V., Horner, R. H., & Odom, S. L. (2015). The role of between-case effect size in conducting, interpreting, and summarizing single-case research (NCER 2015-002). National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. http://ies.ed.gov/
Shadish, W. R., & Sullivan, K. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971–980. https://doi.org/10.3758/s13428-011-0111-y.
Smith, J. D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17(4), 510–550. https://doi.org/10.1037/a0029312.
Tekin-Iftar, E. (2018). Eğitim ve davranış bilimlerinde tek-denekli araştırmalar [Single-case research in education and behavior sciences] (2nd ed.). Anı Yayıncılık.
Tummers, B. (2005-2006). DataThief III manual v. 1.1. DataThief III-Software. http://www.datathief.org/DatathiefManual.pdf
Wichnick-Gillis, A. M., Vener, S. M., & Poulson, C. L. (2019). Script fading for children with autism: Generalization of social initiation skills from school to home. Journal of Applied Behavior Analysis, 52(2), 451–466. https://doi.org/10.1002/jaba.534.
Zimmerman, K. N., Pustejovsky, J. E., Ledford, J. R., Barton, E. E., Severini, K. E., & Lloyd, B. P. (2018). Single-case synthesis tools II: Comparing quantitative outcome measures. Research in Developmental Disabilities, 79, 65–76. https://doi.org/10.1016/j.ridd.2018.02.001.
Acknowledgements
The authors thank Dr. Elif Tekin Iftar for her useful comments.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Approval
Only published data sets were used for the analyses.
Conflict of interest
There is no conflict of interest for the article.
Data availability
All original documents can be provided by the corresponding author.
Funding
There is no funding for this study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Aydin, O., Yassikaya, M.Y. Validity and Reliability Analysis of the PlotDigitizer Software Program for Data Extraction from Single-Case Graphs. Perspect Behav Sci 45, 239–257 (2022). https://doi.org/10.1007/s40614-021-00284-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40614-021-00284-0