Full length article
A meta-analysis of virtual reality training programs

https://doi.org/10.1016/j.chb.2021.106808Get rights and content

Highlights

  • •

    We perform a meta-analysis of controlled experimental studies that assesses VR programs for education and training.

  • •

    VR programs produce better outcomes than tested alternatives.

  • •

    Few moderating effects were significant, including the display hardware, input hardware, and inclusion of game elements.

  • •

    The task-technology fit of the VR training program, however, was a very strong moderator.

Abstract

Virtual reality (VR) is the three-dimensional digital representation of a real or imagined space with interactive capabilities. The application of VR for organizational training purposes has been surrounded by much fanfare; however, mixed results have been provided for the effectiveness of VR training programs, and the attributes of effective VR training programs are still unknown. To address these issues, we perform a meta-analysis of controlled experimental studies that tests the effectiveness of VR training programs. We obtain an estimate of the overall effectiveness of VR training programs, and we identify features of VR training programs that systematically produce improved results. Our meta-analytic findings support that VR training programs produce better outcomes than tested alternatives. The results also show that few moderating effects were significant. The applied display hardware, input hardware, and inclusion of game attributes had non-significant moderating effects; however, task-technology fit and aspects of the research design did influence results. We suggest that task-technology fit theory is an essential paradigm for understanding VR training programs, and no set of VR technologies is “best” across all contexts. Future research should continue studying all types of VR training programs, and authors should more strongly integrate research and theory on employee training and development.

Section snippets

Theoretical background and hypothesis development

VR is defined as the three-dimensional digital representation of a real or imagined space with interactive capabilities (Cruz-Neira et al., 1993; Steuer, 1992; Zyda, 2005). Several notes should be made about this definition. First, users navigate their three-dimensional digital environments using digital representations called avatars. Users may view their avatar from a third-person perspective, or they may be completely unaware of their avatar's appearance by taking a first-person perspective (

Method

We proposed that VR training programs are more effective than alternative training approaches, and we identified many possible moderating characteristics. To best test these proposals, we conduct a meta-analysis of controlled experimental investigations on VR training programs to determine if these training programs produce greater effects, overall, than alternative training programs. We code and analyze attributes of the sources to determine whether specific characteristics alter observed

Publication bias and outlier analyses

Table 1 includes indices of publication bias. The overall fail-safe k was 21,604, and all but two fail-safe k were larger than 300 when restricted to specific significant study/source characteristics. According to prior guidelines (Orwin, 1983; Rothstein et al., 2005), these fail-safe k values are sufficiently large and the meta-analytic effects can be considered robust.

Egger's test, trim-and-fill method, and weight-function analyses signified publication biases in the overall and many subgroup

Discussion

Our results supported that VR training programs perform better than relative comparisons, producing results that are, on average, half a standard deviation better. Our results also showed, however, that only some tested effects significantly influenced the observed results. Display hardware, input hardware, and game attributes did not significantly influence results. The trained outcome did not significantly influence results, whether differentiated by type or complexity. Likewise, the

Conclusion

We supported that VR training programs are, on average, more effective than alternative training programs, but very few boundary conditions significantly influenced these results. We suggest that these repeated null results provide strong support for the overall effectiveness of VR training programs, and they also indicate that no one type of VR training program is superior. Instead, any attribute of VR training programs could be beneficial in the appropriate circumstance. We call for many

Credit author statement

All authors agree to their inclusion and order of the authorship as listed on the title page. All authors were involved in the writing and/or revisions of the current work, which merits their authorship. The first two authors were involved with the methodology and statistical analyses.

References (130)

  • J. Hammick et al.

    Do shy people feel less communication apprehension online? The effects of virtual reality on the relationship between personality characteristics and communication outcomes

    Computers in Human Behavior

    (2014)
  • N.J. Hogle et al.

    Documenting a learning curve and test-retest reliability of two tasks on a virtual reality training simulator in laparoscopic surgery

    Journal of Surgical Education

    (2007)
  • M.C. Howard

    A meta-analysis and systematic literature review of virtual reality rehabilitation programs

    Computers in Human Behavior

    (2017)
  • M.C. Howard et al.

    A meta-analysis of virtual reality training programs for social skill development

    Computers & Education

    (2020)
  • T. Kohler et al.

    Avatar-based innovation: Consequences of the virtual co-creation experience

    Computers in Human Behavior

    (2011)
  • S.H. Lee et al.

    Assessing Google Cardboard virtual reality as a content delivery system in business classrooms

    The Journal of Education for Business

    (2017)
  • A. Moglia et al.

    A systematic review of virtual reality simulators for robot-assisted surgery

    European Urology

    (2016)
  • C.D. Murray et al.

    Absorption, dissociation, locus of control and presence in virtual reality

    Computers in Human Behavior

    (2007)
  • Z. Pan et al.

    Virtual reality and mixed reality for virtual learning environments

    Computers & Graphics

    (2006)
  • N. Pellas

    The influence of computer self-efficacy, metacognitive self-regulation and self-esteem on student engagement in online learning programs: Evidence from the virtual world of second life

    Computers in Human Behavior

    (2014)
  • T.G. Plante et al.

    Might virtual reality promote the mood benefits of exercise?

    Computers in Human Behavior

    (2003)
  • A. Alaraj et al.

    Virtual reality training in neurosurgery: Review of current status and future applications

    Surgical Neurology International

    (2011)
  • E. Ammenwerth et al.

    IT-adoption and the interaction of task, technology and individuals: A fit framework and a case study

    BMC Medical Informatics and Decision Making

    (2006)
  • A. Auerbach

    Making decisions under stress: Implications for individual and team training

    Personnel Psychology

    (1999)
  • J. Au et al.

    Improving fluid intelligence with training on working memory: A meta-analysis

    Psychonomic Bulletin & Review

    (2015)
  • T.T. Baldwin et al.

    Transfer of training: A review and directions for future research

    Personnel Psychology

    (1988)
  • P.N.A. Barata et al.

    Consolidating learning in power systems: Virtual reality applied to the study of the operation of electric power transformers

    IEEE Transactions on Education

    (2015)
  • E. Basaeed et al.

    Learner-centric context-aware mobile learning

    IEEE Multidisciplinary Engineering Education Magazine

    (2007)
  • W. Bedwell et al.

    Toward a taxonomy linking game attributes to learning: An empirical study

    Simulation & Gaming

    (2012)
  • K.K. Bhagat et al.

    A cost-effective interactive 3D virtual reality system applied to military live firing training

    Virtual Reality

    (2016)
  • B.D. Blume et al.

    Transfer of training: A meta-analytic review

    Journal of Management

    (2010)
  • M. Borenstein et al.

    Introduction to meta-analysis

    (2011)
  • F.A. Bosco et al.

    Correlational effect size benchmarks

    Journal of Applied Psychology

    (2015)
  • P. Boxer et al.

    Video games do indeed influence children and adolescents’ aggression, prosocial behavior, and academic performance: A clearer reading of Ferguson (2015)

    Perspectives on Psychological Science

    (2015)
  • M. Butavicius et al.

    Evaluation of a virtual reality parachute training simulator: Assessing learning in an off-course augmented feedback training schedule

    The International Journal of Aviation Psychology

    (2012)
  • C.L. Camp et al.

    Improving resident performance in knee arthroscopy: A prospective value assessment of simulators and cadaveric skills laboratories

    JBJS

    (2016)
  • M. Carter et al.

    The effectiveness of error management training with working‐aged adults

    Personnel Psychology

    (2010)
  • R.K. Champney et al.

    An examination of virtual environment training fidelity on training effectiveness

    International Journal of Learning Technology

    (2017)
  • M. Cheung

    Meta-analysis: A structural equation modeling approach

    (2015)
  • M. Cheung et al.

    Random‐effects models for meta‐analytic structural equation modeling: Review, issues, and illustrations

    Research Synthesis Methods

    (2016)
  • N.J. Cohen et al.

    Different memory systems underlying acquisition of procedural and declarative knowledge

    Annals of the New York Academy of Sciences

    (1985)
  • M. Cordeil et al.

    Immersive collaborative analysis of network connectivity: Cave-style or head-mounted display?

    IEEE Transactions on Visualization and Computer Graphics

    (2017)
  • C. Cruz-Neira et al.

    Surround-screen projection-based virtual reality: The design and implementation of the CAVE

    Proceedings of the 20th annual conference on computer graphics and interactive techniques

    (1993)
  • D.R. Dalton et al.

    Revisiting the file drawer problem in meta‐analysis: An assessment of published and nonpublished correlation matrices

    Personnel Psychology

    (2012)
  • T.E. Doyle et al.

    Closing the design loop in first-year engineering: Modelling and simulation for iterative design

    (2015)
  • L.S. Ferro et al.

    Towards personalised, gamified systems: An investigation into game design, personality and player typologies

    (2013)
  • L. Freina et al.

    A literature review on immersive virtual reality in education: State of the art and perspectives

    (2015)
  • F. Ganier et al.

    Evaluation of procedural learning transfer from a virtual environment to a real situation: A case study on tank maintenance training

    Ergonomics

    (2014)
  • R. Garris et al.

    Games, motivation, and learning: A research and practice model

    Simulation & Gaming

    (2002)
  • N. Gavish et al.

    Evaluating virtual reality and augmented reality training for industrial maintenance and asembly tasks

    Interactive Learning Environments

    (2015)

    • Cited by (33)

    • Study of motivation and engagement for chemical laboratory safety training with VR serious game

      2023, Safety Science

    • Like being there: How the soft VR technology could change consumers’ initial decision-making

      2023, Computers in Human Behavior

    • A Cross-Sectional Analysis of a Novel Virtual Reality Tool for Hotspotting

      2023, Clinical Simulation in Nursing

    • Virtual reality: A review and a new framework for integrated adoption

      2024, International Journal of Consumer Studies

    • Virtual reality-based online learning system adoption: a research framework and empirical study

      2024, Online Information Review

    • Visualization Techniques in VR for Vocational Education: Comparison of Realism and Diegesis on Performance, Memory, Perception and Perceived Usability

      2024, Communications in Computer and Information Science
    View all citing articles on Scopus
    View full text
    © 2021 Elsevier Ltd. All rights reserved.