Skip to main content
SpringerLink
Log in

A scoping review of the relationship between students’ ICT and performance in mathematics and science in the PISA data

  • Published:
Social Psychology of Education Aims and scope Submit manuscript

Abstract

This scoping literature review examines the relationship between ICT and performance scores in mathematics and science for students around the world included in the PISA assessment. In this review we examined 25 publications and showed that the relationship between ICT and academic achievement is not consistent. The different types of ICT revealed different relationships with performance, depending on the subject and country of the students being examined. Although there is a lack of overall consensus, it seems that moderate use, rather than high or no use of ICT, can positively predict academic scores. Although autonomy, interest, and use of ICT as a topic in conversations have been less scrutinized by researchers, they seem to positively predict both mathematics and science scores in 15-year-old students. Implications, limitations, and recommendations are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Achen, C. H. (2005). Let's put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science, 22(4), 327–339.

    Article  Google Scholar 

  • Agasisti, T., Gil-Izquierdo, M., & Han, S. W. (2017). ICT use at home for school-related tasks: What is the effect on a student’s achievement? Empirical evidence from OECD PISA data. Retrieved from https://mpra.ub.uni-muenchen.de/81343/1/MPRA_paper_81343.pdf.

  • Areepattamannil, S., & Santos, I. M. (2019). Adolescent students’ perceived information and communication technology (ICT) competence and autonomy: Examining links to dispositions toward science in 42 countries. Computers in Human Behavior, 98, 50–58.

    Article  Google Scholar 

  • Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616.

    Article  Google Scholar 

  • Bresnahan, T., & Yin, P. L. (2017). Adoption of new information and communications technologies in the workplace today. Innovation Policy and the Economy, 17(1), 95–124.

    Article  Google Scholar 

  • Bulut, O., & Cutumisu, M. (2018). When technology does not add up: ICT use negatively predicts mathematics and science achievement for Finnish and Turkish students in PISA 2012. Journal of Educational Multimedia and Hypermedia (JEMH), 27(1), 25–42. Waynesville, NC USA: Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/178514/.

  • C.D. Howe Institute. (2018). Measuring indigenous education outcomes key to progress. Media Release. Retrieved from https://www.cdhowe.org/media-release/measuring-indigenous-education-outcomes-key-progress.

  • Dall, A. (2011). Is PISA counter-productive to building successful educational systems? Social Alternatives, 30(4), 10.

    Google Scholar 

  • Davie, S. (2017). Nurture joy of learning. The straits times. Retrieved from https://www.straitstimes.com/singapore/education/nurture-joy-of-learning.

  • Delen, E., & Bulut, O. (2011). The relationship between students' exposure to technology and their achievement in science and math. The Turkish Online Journal of Educational Technology, 10, 311–317.

    Google Scholar 

  • Ercikan, K., Roth, W. M., & Asil, M. (2015). Cautions about inferences from international assessments: The case of PISA 2009. Teachers College Record, 117(1), n1.

    Google Scholar 

  • Fuentes, M. D. C., & Gutiérrez, J. J. T. (2012). Does ICT improve Spanish students’ academic performance? In Investigaciones de economía de la educación [número 7, julio 2012] (pp. 955–975). Asociación de Economía de la Educación.

  • Gamazo, A., Martínez-Abad, F., Olmos-Migueláñez, S., & Rodríguez-Conde, M. J. (2018). Evaluación de factoresrelacionados con la eficacia escolar en PISA 2015. Unanálisismultinivel 1 Assessment of factors related to school effectiveness in PISA 2015. A multilevel analysis. Revista de educación, 379, 56–84.

    Google Scholar 

  • Giacquinta, J. B., Bauer, J. A., & Levin, J. E. (1993). Beyond technology's promise: An examination of children's educational computing at home. Cambridge: Cambridge University Press.

    Google Scholar 

  • Goldhaber, D. D., & Brewer, D. J. (2000). Does teacher certification matter? High school teacher certification status and student achievement. Educational Evaluation and Policy Analysis, 22(2), 129–145.

    Article  Google Scholar 

  • Gorur, R., & Wu, M. (2015). Leaning too far? PISA, policy and Australia's ‘top five ‘ambitions. Discourse: Studies in the Cultural Politics of Education, 36(5), 647–664.

    Google Scholar 

  • Grey, S., & Morris, P. (2018). PISA: Multiple ‘truths’ and mediatised global governance. Comparative Education, 54(2), 109–131.

    Article  Google Scholar 

  • Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170–179.

    Article  Google Scholar 

  • Hatos, A. (2020). Is using ICT at home good or bad for learning? A cross-country comparison of the impact of home use of ICT for entertainment and learning on PISA 2015 Science test results.

  • Hu, X., Gong, Y., Lai, C., & Leung, F. K. (2018). The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A multilevel analysis. Computers and Education, 125, 1–13.

    Article  Google Scholar 

  • Hung, C. Y., Sun, J. C. Y., & Yu, P. T. (2015). The benefits of a challenge: Student motivation and flow experience in tablet-PC-game-based learning. Interactive Learning Environments, 23(2), 172–190.

    Article  Google Scholar 

  • Juhaňák, L., Zounek, J., Záleská, K., Bárta, O., & Vlčková, K. (2018). The Relationship between Students' ICT Use and their school performance: Evidence from PISA 2015 in the Czech Republic. Orbisscholae, 12(2), 37–64.

    Google Scholar 

  • Koğar, E. Y. (2019). The investigation of the relationship between mathematics and science literacy and information and communication technology variables. International Electronic Journal of Elementary Education, 11(3), 257–271.

    Google Scholar 

  • Kubiatko, M., & Vlckova, K. (2010). The relationship between ICT use and science knowledge for Czech students: A secondary analysis of PISA 2006. International Journal of Science and Mathematics Education, 8(3), 523–543.

    Article  Google Scholar 

  • Kunina-Habenicht, O., & Goldhammer, F. (2020). ICT engagement: A new construct and its assessment in PISA 2015. Large-Scale Assessments in Education, 8, 1–21.

    Article  Google Scholar 

  • Law, N., Pelgrum, W., & Plomp, T. (2008). Pedagogy and ICT use in schools around the world: Findings from the IEA SITES 2006 study. Hong Kong: Springer.

    Book  Google Scholar 

  • Lei, J. (2010). Quantity versus quality: A new approach to examine the relationship between technology use and student outcomes. British Journal of Educational Technology, 41(3), 455–472.

    Article  Google Scholar 

  • Lei, J., & Zhao, Y. (2007). Technology uses and student achievement: A longitudinal study. Computers and Education, 49(2), 284–296.

    Article  Google Scholar 

  • LeRoy, B. W., Samuel, P., Deluca, M., & Evans, P. (2018). Students with special educational needs within PISA. Assessment in Education: Principles, Policy and Practice, 26, 1–11.

    Google Scholar 

  • Lewis, T. D., Burks, B. A., Thompson, C. N., & Austin, S. M. (2019). Technology impact on K-12 education. In Diverse learning opportunities through technology-based curriculum design (pp. 69–95). IGI Global.

  • Luu, K., & Freeman, J. G. (2011). An analysis of the relationship between information and communication technology (ICT) and scientific literacy in Canada and Australia. Computers and Education, 56(4), 1072–1082.

    Article  Google Scholar 

  • Martínez-Abad, F., Gamazo, A., & Rodríguez-Conde, M. J. (2018b). Big data in education: Detection of ICT factors associated with school effectiveness with data mining techniques. In Proceedings of the sixth international conference on technological ecosystems for enhancing multiculturality (pp. 145–150). ACM.

  • Meggiolaro, S. (2018). Information and communication technologies use, gender and mathematics achievement: Evidence from Italy. Social Psychology of Education, 21(2), 497–516.

    Article  Google Scholar 

  • Meng, L., Qiu, C., & Boyd-Wilson, B. (2018). Measurement invariance of the ICT engagement construct and its association with students’ performance in China and Germany: Evidence from PISA 2015 data. British Journal of Educational Technology, 50, 3233–3251.

    Article  Google Scholar 

  • Murphy, S. (2010). The pull of PISA: Uncertainty, influence, and ignorance. Inter-American Journal of Education for Democracy, 3(1), 27–44.

    Google Scholar 

  • Novak, J., Purta, M., Marciniak, T., Ignatowicz, K., Rozenbaum, K., & Yearwood, K., (2018). The rise of digital challengers: How digitization can become the next growth engine for Central and Eastern Europe. McKinsey & Company.

  • OECD. (2002). PISA 2000 technical report. Paris: OECD Publishing. Retrieved from https://www.oecd.org/pisa/data/33688233.pdf.

  • OECD. (2005). PISA 2003 technical report. Paris: OECD Publishing. Retrieved from https://www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/35188570.pdf.

  • OECD. (2009). PISA 2006 technical report. Paris: OECD Publishing. Retrieved from https://www.educacionyfp.gob.es/inee/dam/jcr:921a96c8-ac4a-4a50-9f20-191849cd5fa6/42025182.pdf.

  • OECD. (2014a). PISA 2012 technical report. Paris: OECD Publishing. Retrieved from https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf.

  • OECD. (2014b). PISA 2015 ICT familiarity questionnaire. Paris: OECD Publishing. Retrieved from https://www.oecd.org/pisa/data/CY6_QST_MS_ICQ_Final.pdf.

  • OECD. (2017). PISA 2015 technical report. Paris: OECD Publishing. Retrieved from https://www.oecd.org/pisa/sitedocument/PISA-2015-technical-report-final.pdf.

  • OECD (2020a). About OECD. Retrieved from https://www.oecd.org/about/.

  • OECD (2020b). OECD FAQ. Retrieved from https://www.oecd.org/pisa/pisafaq/.

  • Özberk, E. H., Kabasakal, K. A., & Öztürk, N. B. (2017). Investigating the factors affecting Turkish students’ PISA 2012 mathematics achievement using hierarchical linear modeling PISA 2012. HacettepeÜniversitesi Journal of Education, 32(3), 544–559. https://doi.org/10.16986/HUJE.2017026950.

    Article  Google Scholar 

  • Papanastasiou, E. C., Zembylas, M., & Vrasidas, C. (2003). Can computer use hurt science achievement? The USA results from PISA. Journal of Science Education and Technology, 12(3), 325–332.

    Article  Google Scholar 

  • Petko, D., Cantieni, A., & Prasse, D. (2017). Perceived quality of educational technology matters: A secondary analysis of students' ICT use, ICT-related attitudes, and PISA 2012 test scores. Journal of Educational Computing Research, 54(8), 1070–1091.

    Article  Google Scholar 

  • Pham, M. T., Rajić, A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., & McEwen, S. A. (2014). A scoping review of scoping reviews: Advancing the approach and enhancing the consistency. Research Synthesis Methods, 5(4), 371–385.

    Article  Google Scholar 

  • Piaget, J. (1952). The origins of intelligence in children (vol. 8, No. 5, p. 18). New York: International Universities Press.

  • Ray, A., & Margaret, W. (Eds.). (2003). PISA programme for international student assessment (PISA) PISA 2000 technical report: PISA 2000 technical report. OECD Publishing.

  • Reboot Foundation. (2019). Does educational technology help students learn? An analysis of the connection between digital devices and learning. Retrieved from https://reboot-foundation.org/does-educational-technology-help-students-learn/.

  • Rieckmann, M. (2017). Education for sustainable development goals: Learning objectives. UNESCO Publishing.

  • Rodrigues, M., & Biagi, F. (2017). Digital technologies and learning outcomes of students from low socio-economic background: An analysis of PISA 2015. JRC Science for Policy Report.

  • Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68–78.

    Article  Google Scholar 

  • Sjøberg, S. (2017). PISA testing-a global educational race? Europhysics News, 48(4), 17–20.

    Article  Google Scholar 

  • Skryabin, M., Zhang, J., Liu, L., & Zhang, D. (2015). How the ICT development level and usage influence student achievement in reading, mathematics, and science. Computers and Education, 85, 49–58.

    Article  Google Scholar 

  • Srijamdee, K., & Pholphirul, P. (2020). Does ICT familiarity always help promote educational outcomes? Empirical evidence from PISA-Thailand. Education and Information Technologies, 25, 1–38.

    Article  Google Scholar 

  • Su, M. (2017). The influence of information and communication technology (ICT) on Chinese and Korean students' math achievement in PISA 2015 (Doctoral dissertation, State University of New York at Buffalo).

  • Tan, C. Y., & Hew, K. F. (2018). The impact of digital divides on student mathematics achievement in Confucian heritage cultures: A critical examination using PISA 2012 data. International Journal of Science and Mathematics Education, 17, 1–20.

    Google Scholar 

  • Zhang, D., & Liu, L. (2016). How does ICT use influence students’ achievements in math and science over time? Evidence from PISA 2000 to 2012. Eurasia Journal of Mathematics, Science and Technology Education, 12(9), 2431–2449.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the editor and the reviewers who provided us with helpful and thorough feedback and suggestions that improved this manuscript. We would also like to thank the following granting agencies for their support: Social Sciences and Humanities Research Council of Canada—Insight Development Grant (SSHRC IDG) RES0034954 and Insight Grant (SSHRC IG) RES0048110, the Natural Sciences and Engineering Research Council (NSERC DG) RES0043209, and the Killam Cornerstone Operating Grant RES0043207.

Author information

Authors and Affiliations

  1. Centre for Research in Applied Measurement and Evaluation, Department of Educational Psychology, University of Alberta, 6-102 Education North, Faculty of Education, 11210 - 87 Avenue, Edmonton, AB, T6G 2G5, Canada

    Bryce Odell, Maria Cutumisu & Mark Gierl

Authors
  1. Bryce Odell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria Cutumisu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark Gierl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria Cutumisu.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Odell, B., Cutumisu, M. & Gierl, M. A scoping review of the relationship between students’ ICT and performance in mathematics and science in the PISA data. Soc Psychol Educ 23, 1449–1481 (2020). https://doi.org/10.1007/s11218-020-09591-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11218-020-09591-x

Keywords

Search

Navigation