Abstract
This study expands on Keiser and Arthur’s (2021) meta-analysis of the after-action review (AAR), or debrief, by examining six additional task and training characteristics that contribute to or attenuate its effectiveness. The findings based on a bare-bones meta-analysis of results from 83 studies (134 ds [955 teams; 4,684 individuals]) indicate that the effectiveness of the AAR (overall d = 0.92) does indeed vary across the pertinent characteristics. The primary impact of this study pertains to the practical implementation of AARs; notably, the findings indicate that the AAR is particularly effective in task environments that are characterized by a combination of high complexity and ambiguity in terms of offering no intrinsic feedback. The types of tasks—often project and decision-making—that more commonly entail these characteristics are frequently used in industries that do not traditionally use the AAR. The results also suggest that more recent variants of the AAR (i.e., a reaction phase, a canned performance review) do not meaningfully add to its effectiveness. These findings are combined with those from prior meta-analyses to derive 11 empirically-based practical guidelines for the use of AARs. In sum, this study highlights the complexity of the AAR that results from the independent and interdependent influence among various components and characteristics, the examination of the effects of novel and ostensibly distinct variants or approaches to AARs, and the extension of AARs to tasks and contexts in which they are less commonly used.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
There is some overlap in the present dataset and Keiser and Arthur’s (2021) meta-analysis in that we aggregate the same effects from the 61 studies also included in Keiser and Arthur. However, the present study provides an update that includes more recent studies. Indeed, the burgeoning nature of the research literature on AARs in healthcare is reflected in the identification of an additional 27 ds from 22 codeable studies published in the healthcare literature after the terminal date for Keiser and Arthur’s search. Furthermore, whereas there is some overlap in data points, the focus of this study is on task and training characteristics that have not been examined in any prior meta-analysis.
References
References marked with an asterisk indicate studies included in the meta-analysis
*Abraham, C., & Kusheleva, N. (2019). Management of pre-eclampsia and eclampsia: A simulation. MedEdPORTAL, 15. https://doi.org/10.15766/mep_2374-8265.10832.
Ahmed, M., Sevdalis, N., Paige, J., Paragi-Gururaja, R., Nestel, D., & Arora, S. (2012). Identifying best practice guidelines for debriefing in surgery: A tri-continental study. American Journal of Surgery, 203, 523–529. https://doi.org/10.1016/j.amjsurg.2011.09.024
Alfieri, L., Nokes-Malach, T. J., & Schunn, C. D. (2013). Learning through case comparisons: A meta-analytic review. Educational Psychologist, 48, 87–113. https://doi.org/10.1080/00461520.2013.775712
*Al-Harthi, R. A., Scott, J. B., Mirza, S. H., Wadhwani, T. K., Dubosky, M. N., & Vines, D. L. (2016). Evaluation of technology-enhanced education and debriefing sessions for teaching high frequency oscillatory ventilation (HFOV) to physicians and respiratory therapists for use as a rescue intervention in adult patients. Respiratory Care, 61, 49.
*Ali, A. A., Miller, E., Ballman, K., Bakas, T., Geis, G., & Ying, J. (2020). The impact of debriefing modalities on nurse practitioner students’ knowledge and leadership skills in managing fatal dysrhythmias: A pilot study. Nurse Education in Practice, 42, 102687. https://doi.org/10.1016/j.nepr.2019.102687
Allen, J. A., Reiter-Palmon, R., Crowe, J., & Scott, C. (2018). Debriefs: Teams learning from doing in context. American Psychologist, 73, 504–516. https://doi.org/10.1037/amp0000246
*Anderson, J. M., Murphy, A. A., Boyle, K. B., Yaeger, K. A., & Halamek, L. P. (2006). Simulating extracorporeal membrane oxygenation emergencies to improve human performance. Part II: Assessment of technical and behavioral skills. Simulation in Healthcare, 1, 228–232. https://doi.org/10.1097/01.SIH.0000243551.01521.74
Annett, J. (1969). Feedback and human behaviour: The effects of knowledge of results, incentives and reinforcement on learning and performance. Penguin Books.
Arthur Jr., W., Bennett Jr., W., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88, 234–245. https://doi.org/10.1037/0021-9010.88.2.234
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice Hall.
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101. https://doi.org/10.2307/2533446
Bell, B. S., Tannenbaum, S. I., Ford, J. K., Noe, R. A., & Kraiger, K. (2017). 100 years of training and development research: What we know and where we should go. Journal of Applied Psychology, 102, 305–323. https://doi.org/10.1037/apl0000142
*Belote, M. J. (2015). A comparative evaluation of the learner centered grading debriefing method in nursing education (Unpublished doctoral dissertation). University of South Florida, Tampa, FL.
*Boet, S., Bould, M. D., Bruppacher, H. R., Desjardins, F., Chandra, D. B., & Naik, V. N. (2011). Looking in the mirror: Self-debriefing versus instructor debriefing for simulated crises. Critical Care Medicine, 39, 1377–1381. https://doi.org/10.1097/CCM.0b013e31820eb8be
*Boet, S., Bould, M. D., Sharma, B., Revees, S., Naik, V. N., Triby, E., & Grantcharov, T. (2013). Within-team debriefing versus instructor-led debriefing for simulation-based education: A randomized controlled trial. Annals of Surgery, 258, 53–58. https://doi.org/10.1097/SLA.0b013e31829659e4
*Bolinger, A. R., & Stanton, J. V. (2014). The gap between perceived and actual learning from group reflection. Small Group Research, 45, 539–567. https://doi.org/10.1177/1046496414538322
*Bragard, I., Farhat, N., Seghaye, M. C., Karam, O., Neuschwander, A., Shayan, Y., & Schumacher, K. (2019). Effectiveness of a high-fidelity simulation-based training program in managing cardiac arrhythmias in children: A randomized pilot study. Pediatric Emergency Care, 35, 412–418. https://doi.org/10.1097/PEC.0000000000000931
*Campbell, D. M., Barozzino, T., Farrugia, M., & Sgro, M. (2009). High fidelity simulation in neonatal resuscitation. Paediatrics & Child Health, 14, 19–23. https://doi.org/10.1093/pch/14.1.19
Canty, A., & Ripley, B. (2017). boot: Bootstrap R (S-Plus) Functions (R package version 1.3–20) [Computer software]. Retrieved from https://CRAN.R-project.org/package_boot.
*Catanzaro, J. E. (2016). Debriefing for meaningful learning model: Effect on clinical reasoning proficiency in baccalaureate nursing students (Unpublished doctoral dissertation). Northcentral University, Prescott Valley, AZ.
*Chen, J., Bamberger, P. A., Song, Y., & Vashdi, D. R. (2018). The effects of team reflexivity on psychological well-being in manufacturing teams. Journal of Applied Psychology, 103, 443–462. https://doi.org/10.1037/apl0000279
Cheng, A., Eppich, W., Grant, V., Sherbino, J., Zendejas, B., & Cook, D. A. (2014). Debriefing for technology-enhanced simulation: A systematic review and meta-analysis. Medical Education, 48, 657–666. https://doi.org/10.1111/medu.12432
*Chronister, C., & Brown, D. (2012). Comparison of simulation debriefing methods. Clinical Simulation in Nursing, 8, e281–e288. https://doi.org/10.1016/j.ecns.2010.12.005
*Chung, S. P., Cho, J., Park, Y. S., Kang, H. G., Kim, C. W., Song, K. J.,. . . Cho, G. C. (2011). Effects of script-based role play in cardiopulmonary resuscitation team training. Emergency Medicine Journal, 28, 690–694. https://doi.org/10.1136/emj.2009.090605.
Cohen, S. G., & Bailey, D. E. (1997). What makes teams work: Group effectiveness research from the shop floor to the executive suite. Journal of Management, 23, 239–290. https://doi.org/10.1177/014920639702300303
*Coppens, I., Verhaeghe, S., Van Hecke, A., & Beeckman, D. (2018). The effectiveness of crisis resource management and team debriefing in resuscitation education of nursing students: A randomised controlled trial. Journal of Clinical Nursing, 27, 77–85. https://doi.org/10.1111/jocn.13846
Couper, K., Salman, B., Soar, J., Finn, J., & Perkins, G. D. (2013). Debriefing to improve outcomes from critical illness: A systematic review and meta-analysis. Intensive Care Medicine, 39, 1513–1523. https://doi.org/10.1007/s00134-013-2951-7
Dachler, H. P., & Mobley, W. H. (1973). Construct validation of an instrumentality-expectancy-task-goal model of work motivation: Some theoretical boundary conditions. Journal of Applied Psychology, 58, 397–418. https://doi.org/10.1037/h0036297
Dahlke, J. A., & Wiernik, B. M. (2019). psychmeta: An R package for psychometric meta-analysis. Applied Psychological Measurement, 43, 415–416. https://doi.org/10.1177/0146621618795933
Davison, A. C., & Hinkley, D. V. (1997). Bootstrap methods and their applications. Cambridge University Press. https://doi.org/10.1017/CBO9780511802843
De Dreu, C. K., & Weingart, L. R. (2003). Task versus relationship conflict, team performance, and team member satisfaction: A meta-analysis. Journal of Applied Psychology, 88, 741–749. https://doi.org/10.1037/0021-9010.88.4.741
Densen, P. (2011). Challenges and opportunities facing medical education. Transactions of the American Clinical and Climatological Association, 122, 48–58.
Department of the Army. (1993). A leaders guide to after-action reviews (Training Circular 25–20). Washington, DC: Author. Retrieved from https://www.hsdl.org/?view&did_775082.
*DeRue, D. S., Nahrgang, J. D., Hollenbeck, J. R., & Workman, K. (2012). A quasi-experimental study of after-event reviews and leadership development. Journal of Applied Psychology, 97, 997–1015. https://doi.org/10.1037/a0028244
DeShon, R. P., Kozlowski, S. W., Schmidt, A. M., Milner, K. R., & Wiechmann, D. (2004). A multiple-goal, multilevel model of feedback effects on the regulation of individual and team performance. Journal of Applied Psychology, 89, 1035–1056. https://doi.org/10.1037/0021-9010.89.6.1035
*DeVita, M. A., Schaefer, J., Lutz, J., Wang, H., & Dongilli, T. (2005). Improving medical emergency team (MET) performance using a novel curriculum and a computerized human patient simulator. Quality & Safety in Health Care, 14, 326–331. https://doi.org/10.1136/qshc.2004.011148
*Dine, C. J., Gersh, R. E., Leary, M., Riegel, B. J., Bellini, L. M., & Abella, B. S. (2008). Improving cardiopulmonary resuscitation quality and resuscitation training by combining audiovisual feedback and debriefing. Critical Care Medicine, 36, 2817–2822. https://doi.org/10.1097/CCM.0b013e318186fe37
*Dreifuerst, K. T. (2012). Using debriefing for meaningful learning to foster development of clinical reasoning in simulation. The Journal of Nursing Education, 51, 326–333. https://doi.org/10.3928/01484834-20120409-02
*Dusaj, T. K. (2014). A randomized control study comparing outcomes in student nurses who utilize video during simulation debriefing as compared to those who utilize traditional debriefing (Unpublished doctoral dissertation). Rutgers University, New Brunswick, NJ.
Duval, S. J., & Tweedie, R. L. (2000a). A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89–98.
Duval, S., & Tweedie, R. (2000b). Trim and fill: A simple funnel-plot based method of testing and adjusting for publication bias in metaanalysis. Biometrics, 56, 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
*Edelson, D. P., Litzinger, B., Arora, V., Walsh, D., Kim, S., Lauderdale, D. S., . . . Abella, B. S. (2008). Improving in-hospital cardiac arrest process and outcomes with performance debriefing. Archives of Internal Medicine, 168, 1063–1069. https://doi.org/10.1001/archinte.168.10.1063.
Edmondson, A. C. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350–383. https://doi.org/10.2307/2666999
*Edwards, T. F. (2016). An examination of undergraduate sophomore-level nursing students’ personal reflection ability following high-fidelity simulation and video-assisted debriefing (Unpublished doctoral dissertation). Indiana University of Pennsylvania, Indiana, PA.
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629–634. https://doi.org/10.1136/bmj.315.7109.629
*Ellis, S., & Davidi, I. (2005). After-event reviews: Drawing lessons from successful and failed experience. Journal of Applied Psychology, 90, 857–871. https://doi.org/10.1037/0021-9010.90.5.857
*Ellis, S., Mendel, R., & Nir, M. (2006). Learning from successful and failed experience: The moderating role of kind of after-event review. Journal of Applied Psychology, 91, 669–680. https://doi.org/10.1037/0021-9010.91.3.669
*Ellis, S., Mendel, R., & Aloni-Zohar, M. (2009). The effect of accuracy of performance evaluation on learning from experience: The moderating role of after-event reviews. Journal of Applied Social Psychology, 39, 541–563. https://doi.org/10.1111/j.1559-1816.2009.00450.x
*Ellis, S., Ganzach, Y., Castle, E., & Sekely, G. (2010). The effect of filmed versus personal after-event reviews on task performance: The mediating and moderating role of self-efficacy. Journal of Applied Psychology, 95, 122–131. https://doi.org/10.1037/a0017867
Eppich, W., & Cheng, A. (2015). Promoting Excellence and Reflective Learning in Simulation (PEARLS): Development and rationale for a blended approach to health care simulation debriefing. Simulation in Healthcare, 10, 106–115. https://doi.org/10.1097/SIH.0000000000000072
Fanning, R. M., & Gaba, D. M. (2007). The role of debriefing in simulation-based learning. Simulation in Healthcare, 2, 115–125. https://doi.org/10.1097/SIH.0b013e3180315539
*Feather, R. A., Carr, D. E., Garletts, D. M., & Reising, D. L. (2017). Nursing and medical students teaming up: Results of an interprofessional project. Journal of Interprofessional Care, 31, 661–663. https://doi.org/10.1080/13561820.2017.1322563
Fernandez, A. E. (2014). Meta-analysis of simulation debriefing research (Unpublished doctoral dissertation). Walden University, Minneapolis, MN.
Frese, M., & Zapf, D. (1994). Action as the core of work psychology: A German approach. In H. C. Triandis, M. D. Dunnette, & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 4, 2nd ed., pp. 271–340). Consulting Psychologists Press.
*Gardner, A. K., Kosemund, M., Hogg, D., Heymann, A., & Martinez, J. (2017). Setting goals, not just roles: Improving teamwork through goal focused debriefing. American Journal of Surgery, 213, 249–252. https://doi.org/10.1016/j.amjsurg.2016.09.040
*Gaylle, D. (2015). Effects of a mental-health clinical simulation experience using standardized patients and two debriefing styles on prelicensure nursing students’ knowledge, anxiety, and therapeutic communication and psychiatric assessment skills (Unpublished doctoral dissertation). The University of San Francisco, San Francisco, CA.
*Goffman, D., Heo, H., Pardanani, S., Merkatz, I. R., & Bernstein, P. S. (2008, Improving shoulder dystocia management among resident and attending physicians using simulations. American Journal of Obstetrics and Gynecology, 199, 294.e1–294.e5. https://doi.org/10.1016/j.ajog.2008.05.023
*Guimond, M. E., Foreman, S. E., & Werb, M. (2019). Evaluation of an unfolding obstetric experience simulation in an undergraduate nursing program. Nurse Education Today, 79, 124–128. https://doi.org/10.1016/j.nedt.2019.05.003
*Ha, E. H. (2020). Effects of peer-led debriefing using simulation with case-based learning: Written vs. observed debriefing. Nurse Education Today, 84, 124–128. https://doi.org/10.1016/j.nedt.2019.05.003
Hackman, J. R., & Oldham, G. R. (1975). Development of the job diagnostic survey. Journal of Applied Psychology, 60, 159-170. https://doi.org/10.1037/h0076546
*Hamad, G. G., Brown, M. T., & Clavijo-Alvarez, J. A. (2007). Postoperative video debriefing reduces technical errors in laparoscopic surgery. American Journal of Surgery, 194, 110–114. https://doi.org/10.1016/j.amjsurg.2006.10.027
*Hamilton, N. A., Kieninger, A. N., Woodhouse, J., Freeman, B. D., Murray, D., & Klingensmith, M. E. (2012). Video review using a reliable evaluation metric improves team function in high-fidelity simulated trauma resuscitation. Journal of Surgical Education, 69, 428–431. https://doi.org/10.1016/j.jsurg.2011.09.009
Hollenbeck, J. R., Beersma, B., & Schouten, M. E. (2012). Beyond team types and taxonomies: A dimensional scaling conceptualization for team description. Academy of Management Review, 37, 82–106. https://doi.org/10.5465/amr.2010.0181
Huffcutt, A. I., & Arthur Jr., W. (1995). Development of a new outlier statistic for meta-analytic data. Journal of Applied Psychology, 80, 327–334. https://doi.org/10.1037/0021-9010.80.2.327
Hughes, A. M., Gregory, M. E., Joseph, D. L., Sonesh, S. C., Marlow, S. L., Lacerenza, C. N., . . . Salas, E. (2016). Saving lives: A meta-analysis of team training in healthcare. Journal of Applied Psychology, 101, 1266–1304. https://doi.org/10.1037/apl0000120.
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings ((2nd ed.). ed.). Sage. https://doi.org/10.4135/9781412985031
*Janicas, R. D. C. S. V., & Narchi, N. Z. (2019). Evaluation of nursing students’ learning using realistic scenarios with and without debriefing. Revista Latino-Americana de Enfermagem, 27. https://doi.org/10.1590/1518-8345.2936.3187
*Jansson, M. M., Syrjälä, H. P., Ohtonen, P. P., Meriläinen, M. H., Kyngäs, H. A., & Ala-Kokko, T. I. (2017). Longitudinal effects of single-dose simulation education with structured debriefing and verbal feedback on endotracheal suctioning knowledge and skills: A randomized controlled trial. American Journal of Infection Control, 45, 83–85. https://doi.org/10.1016/j.ajic.2016.05.032
*Jarrett, S. M., Glaze, R. M., Schurig, I., Muñoz, G. J., Naber, A. M., McDonald, J. N., . . . Arthur, W., Jr. (2016). The comparative effectiveness of distributed and colocated team after-action reviews. Human Performance, 29, 408–427. https://doi.org/10.1080/08959285.2016.1208662.
Jones, K. J., Crowe, J., Allen, J. A., Skinner, A. M., High, R., Kennel, V., & Reiter-Palmon, R. (2019). The impact of post-fall huddles on repeat fall rates and perceptions of safety culture: A quasi-experimental evaluation of a patient safety demonstration project. BMC Health Services Research, 19, 650. https://doi.org/10.1186/s12913-019-4453-y
Kahn, W. A. (1990). Psychological conditions of personal engagement and disengagement at work. Academy of Management Journal, 33, 692–724. https://doi.org/10.5465/256287
Keiser, N. L., & Arthur Jr., W. (2021). A meta-analysis of the effectiveness of the after-action review (or debrief) and factors that influence to its effectiveness. Journal of Applied Psychology, 106, 1007–1032 https://psycnet.apa.org/doi/10.1037/apl0000821
Kikkawa, Y., & Mavin, T. J. (2017). A review of debriefing practices: Toward a framework for airline pilot debriefing. Aviation Psychology and Applied Human Factors, 7, 42–54. https://doi.org/10.1027/2192-0923/a000114
*Kim, J., & Lee, O. (2020). Effects of a simulation-based education program for nursing students responding to mass casualty incidents: A pre-post intervention study. Nurse Education Today, 85, 104297. https://doi.org/10.1016/j.nedt.2019.104297
Kim, Y. J., & Yoo, J. H. (2020). The utilization of debriefing for simulation in healthcare: A literature review. Nurse Education in Practice, 43, 102698. https://doi.org/10.1016/j.nepr.2020.102698
Kleingeld, A., van Mierlo, H., & Arends, L. (2011). The effect of goal setting on group performance: A meta-analysis. Journal of Applied Psychology, 96, 1289–1304. https://doi.org/10.1037/a0024315
Kluger, A. N., & DeNisi, A. (1996). Effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284. https://doi.org/10.1037/0033-2909.119.2.254
*Kolbe, M., Weiss, M., Grote, G., Knauth, A., Dambach, M., Spahn, D. R., & Grande, B. (2013). TeamGAINS: A tool for structured debriefings for simulation-based team trainings. British Medical Journal Quality & Safety, 22, 541–553. https://doi.org/10.1136/bmjqs-2012-000917
Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311–328. https://doi.org/10.1037/0021-9010.78.2.311
*Kring, J. P. (2004). Communication modality and after action review performance in a distributed immersive virtual environment (Unpublished doctoral dissertation). University of Central Florida, Orlando, FL.
Lacerenza, C. N., Reyes, D. L., Marlow, S. L., Joseph, D. L., & Salas, E. (2017). Leadership training design, delivery, and implementation: A meta-analysis. Journal of Applied Psychology, 102, 1686–1718. https://doi.org/10.1037/apl0000241
*Lee, M. N., Kim, S. J., Kang, K. A., & Kim, S. (2020). Comparing the learning effects of debriefing modalities for the care of premature infants. Nursing & Health Sciences, 22, 243–253. https://doi.org/10.1111/nhs.12662
Levine, A., DeMaria, S., Schwartz, A., & Sim, A. (2013). The comprehensive textbook of healthcare simulation. Springer.
Lipsey, M. W. (2003). Those confounded moderators in meta-analysis: Good, bad, and ugly. Annals of the American Academy of Political and Social Science, 587, 69–81. https://doi.org/10.1177/0002716202250791
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Sage.
Locke, E. A. (1968). Toward a theory of task motivation and incentives. Organizational Behavior and Human Performance, 3, 157–189. https://doi.org/10.1016/0030-5073(68)90004-4
Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. American Psychologist, 57, 705–717. https://doi.org/10.1037/0003-066X.57.9.705
Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions in Psychological Science, 15, 265–268. https://doi.org/10.1111/j.1467-8721.2006.00449.x
Locke, E. A., Latham, G. P., & Erez, M. (1988). The determinants of goal commitment. The Academy of Management Review, 13, 23–39. https://doi.org/10.5465/amr.1988.4306771
Macaskill, P., Walter, S. D., & Irwig, L. (2001). A comparison of methods to detect publication bias in meta-analysis. Statistics in Medicine, 20, 641–654. https://doi.org/10.1002/sim.698
*Mariani, B., Cantrell, M. A., Meakim, C., Prieto, P., & Dreifuerst, K. T. (2013). Structured debriefing and students’ clinical judgment abilities in simulation. Clinical Simulation in Nursing, 9, e147–e155. https://doi.org/10.1016/j.ecns.2011.11.009
*Marken, P. A., Zimmerman, C., Kennedy, C., Schremmer, R., & Smith, K. V. (2010). Human simulators and standardized patients to teach difficult conversations to interprofessional health care teams. American Journal of Pharmaceutical Education, 74, 120. https://doi.org/10.5688/aj7407120
Mathieu, J. E., Hollenbeck, J. R., van Knippenberg, D., & Ilgen, D. R. (2017). A century of work teams in the Journal of Applied Psychology. Journal of Applied Psychology, 102, 452–467. https://doi.org/10.1037/apl0000128
*Mikrogianakis, A., Osmond, M. H., Nuth, J. E., Shephard, A., Gaboury, I., & Jabbour, M. (2008). Evaluation of a multidisciplinary pediatric mock trauma code educational initiative: A pilot study. The Journal of Trauma, 64, 761–767. https://doi.org/10.1097/TA.0b013e3180341ff8
*Morgan, P. J., Cleave-Hogg, D., McIlroy, J., & Devitt, J. H. (2002). Simulation technology: A comparison of experiential and visual learning for undergraduate medical students. Anesthesiology, 96, 10–16. https://doi.org/10.1097/00000542-200201000-00008
*Morgan, P. J., Cleave-Hogg, D., Desousa, S., & Lam-McCulloch, J. (2006). Applying theory to practice in undergraduate education using high fidelity simulation. Medical Teacher, 28, e10–e15. https://doi.org/10.1080/01421590600568488
*Morgan, P. J., Tarshis, J., LeBlanc, V., Cleave-Hogg, D., DeSousa, S., Haley, M. F., . . . Law, J. A. (2009). Efficacy of high-fidelity simulation debriefing on the performance of practicing anaesthetists in simulated scenarios. British Journal of Anaesthesia, 103, 531–537. https://doi.org/10.1093/bja/aep222.
Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105–125. https://doi.org/10.1037/1082-989X.7.1.105
Morrison, J. E., & Meliza, L. L. (1999). Foundations of the after action review process (Special Report 42). U.S. Army Research Institute for the Behavioral and Social Sciences.
Muñoz, G. J., Cortéz, D. A., Álvarez, C. B., Raggio, J. A., Concha, A., Rojas, F. I., . . . Rodriguez, S. (2020). After-action reviews and long-term performance: An experimental examination in the context of an emergency simulation. Human Factors. https://doi.org/10.1177/0018720820958848
*Nguyen, N., Watson, W. D., & Dominguez, E. (2019). Simulation-based communication training for general surgery and obstetrics and gynecology residents. Journal of Surgical Education, 76, 856–863. https://doi.org/10.1016/j.jsurg.2018.10.014
*Niemants, J., Hanchard, B., Gerard, V., & Horlait, G. (2016). Comparing effectiveness of video-assisted oral debriefing versus oral debriefing alone during human resuscitation simulation: A randomized trial. Resuscitation, 106, e87. https://doi.org/10.1016/j.resuscitation.2016.07.209
Noe, R. A. (2013). Employee training and development (6th ed.). Irwin-McGraw.
Nunnally, J. (1978). Psychometric methods. McGraw-Hill.
*Nwokorie, N., Svoboda, D., Rovito, D. K., & Krugman, S. D. (2012). Effect of focused debriefing on team communication skills. Hospital Pediatrics, 2, 221–227. https://doi.org/10.1542/hpeds.2011-0006-2
*O’Donnell, J. M., Goode, J. S., Jr., Henker, R. A., Kelsey, S., Bircher, N., Peele, P., . . . Sutton-Tyrrell, K. (2012). An ergonomic protocol for patient transfer that can be successfully taught using simulation methods. Clinical Simulation in Nursing, 8, e3–e14. https://doi.org/10.1016/j.ecns.2010.05.003.
*Oden, K. B. (2008). Distributed team training: Effective team feedback (Unpublished doctoral dissertation). University of Central Florida, Orlando, FL.
*Odongkara, B., Tylleskär, T., Pejovic, N., Achora, V., Mukunya, D., Ndeezi, G., ... & Nankabirwa, V. (2020). Adding video-debriefing to helping-babies-breathe training enhanced retention of neonatal resuscitation knowledge and skills among health workers in Uganda: A cluster randomized trial. Global Health Action, 13, 1743496. https://doi.org/10.1080/16549716.2020.1743496.
*Olson, S. L. (2013). Debriefing after high-fidelity simulation and knowledge retention: A quasi-experimental study (Unpublished doctoral dissertation). Capella University, Minneapolis, MN.
*Ostovar, S., Allahbakhshian, A., Gholizadeh, L., Dizaji, S. L., Sarbakhsh, P., & Ghahramanian, A. (2019). Comparing effects of debriefing methods on psycho-motor skills, self-confidence and satisfaction in novice nursing students: A quasi-experimental study. Maedica, 14, 15–21. https://doi.org/10.26574/maedica.2019.14.1.15
*Paige, J., Garbee, D., Yu, Q., Kiselov, V., Rusnak, V., & Detiege, P. (2019). Moving along: Team training for emergency room trauma transfers (T2ERT2). Journal of Surgical Education, 76, 1402–1412. https://doi.org/10.1016/j.jsurg.2019.03.013
*Palmer, E., Labant, A. L., Edwards, T. F., & Boothby, J. (2019). A collaborative partnership for improving newborn safety: Using simulation for neonatal resuscitation training. The Journal of Continuing Education in Nursing, 50, 319–324. https://doi.org/10.3928/00220124-20190612-07
Pearson, K. (1904/1948). On the theory of contingency and its relation to association and normal correlation. Karl Pearson’s early statistical papers (pp. 443–475). Cambridge University Press.
Pearson, M., & Smith, D. (1985). Debriefing in experience-based learning. In D. Boud, R. Keogh, & D. Walker (Eds.), Reflection: Turning experience into learning (pp. 69–84). Kogan Page.
*Prakash, S., Bihari, S., Laver, R., Chandran, G., Kerr, L., Schuwirth, L., & Bersten, A. (2020). Prospective randomized controlled trial of video-versus recall-assisted reflection in simulation-based teaching on acquisition and retention of airway skills among trainees intubating critically ill patients. Critical Care Medicine, 48, 1265–1270. https://doi.org/10.1097/CCM.0000000000004448
*Qudrat-Ullah, H. (2007). Debriefing can reduce misperceptions of feedback: The case of renewable resource management. Simulation & Gaming, 38, 382–397. https://doi.org/10.1177/1046878107300669
*Reeves, N. E., Waite, M. C., Tuttle, N., & Bialocerkowski, A. (2019). Simulated patient contributions to enhancing exercise physiology student clinical assessment skills. Advances in Simulation, 4, 15. https://doi.org/10.1186/s41077-019-0097-6
*Robertson, B., Schumacher, L., Gosman, G., Kanfer, R., Kelley, M., & DeVita, M. (2009). Simulation-based crisis team training for multidisciplinary obstetric providers. Simulation in Healthcare, 4, 77–83. https://doi.org/10.1097/SIH.0b013e31819171cd
*Roh, Y. S., Kelly, M., & Ha, E. H. (2016). Comparison of instructor-led versus peer-led debriefing in nursing students. Nursing & Health Sciences, 18, 238–245. https://doi.org/10.1111/nhs.12259
*Ross, M. E. T., Bryan, J. L., Thomas, K. L., Asghar-Ali, A. A., & Pickens, S. L. (2020). Elder abuse education using standardized patient simulation in an undergraduate nursing program. Journal of Nursing Education, 59, 331–335. https://doi.org/10.3928/01484834-20200520-06
Rudolph, J. W., Simon, R., Dufresne, R. L., & Raemer, D. B. (2006). There’s no such thing as “nonjudgmental” debriefing: A theory and method for debriefing with good judgment. Simulation in Healthcare, 1, 49–55. https://doi.org/10.1097/01266021-200600110-00006
*Ryoo, E. N., & Ha, E. H. (2015). The importance of debriefing in simulation-based learning: Comparison between debriefing and no debriefing. Computers, Informatics, Nursing: CIN, 33, 538–545.
*Saiva, A., Abdool, P. S., Naismith, L. M., & Nirula, L. (2020). An immersive simulation to build empathy for geriatric patients with co-occurring physical and mental illness. Academic Psychiatry. https://doi.org/10.1007/s40596-020-01233-w
Salas, E., Klein, C., King, H., Salisbury, M., Augenstein, J. S., Birnbach, D. J., Robinson, D. W., & Upshaw, C. (2008). Debriefing medical teams: 12 evidence-based practices and tips. Joint Commission Journal on Quality and Patient Safety, 34, 518–527.
Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The science of training and development in organizations: What matters in practice. Psychological Science in the Public Interest, 13, 74–101. https://doi.org/10.1177/1529100612436661
*Sanders, M. J. (2005). The effect of immediate feedback and after-action reviews (AARs) on learning, retention and transfer (Unpublished master’s thesis). University of Central Florida, Orlando, FL.
Sawyer, T. L., & Deering, S. (2013). Adaptation of the U.S. Army’s after-action review for simulation debriefing in healthcare. Simulation in Healthcare, 8, 388–397. https://doi.org/10.1097/SIH.0b013e31829ac85c
*Sawyer, T., Sierocka-Castaneda, A., Chan, D., Berg, B., Lustik, M., & Thompson, M. (2012). The effectiveness of video-assisted debriefing versus oral debriefing alone at improving neonatal resuscitation performance: A randomized trial. Simulation in Healthcare, 7, 213–221. https://doi.org/10.1097/SIH.0b013e3182578eae
Sawyer, T., Eppich, W., Brett-Fleegler, M., Grant, V., & Cheng, A. (2016). More than one way to debrief: A critical review of healthcare simulation debriefing methods. Simulation in Healthcare, 11, 209–217. https://doi.org/10.1097/SIH.0000000000000148
Schank, R. (1986). Explanation patterns: Understanding mechanically and creatively. Erlbaum.
Schein, E. H., & Bennis, W. G. (1965). Personal and organizational change through group methods: The laboratory approach. Wiley.
Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings ((3rd ed.). ed.). Sage. https://doi.org/10.4135/9781483398105
Scott, G., Leritz, L. E., & Mumford, M. D. (2004). The effectiveness of creativity training: A quantitative review. Creativity Research Journal, 16, 361–388. https://doi.org/10.1080/10400410409534549
Senge, P. (1994). The fifth discipline fieldbook: Strategies and tools for building a learning organization. Crown Business.
*Shailaja, S., Hilda, S. S., Pinto, P. A., D'Cunha, R. J., Mahmood, L. S., & Hegde, R. B. (2019). Evaluation of resident satisfaction and change in knowledge following use of high-fidelity simulation teaching for anaesthesia residents. Indian Journal of Anaesthesia, 63, 908–914. https://doi.org/10.4103/ija.IJA_133_19
*Shi, D., Lu, H., Wang, H., Bao, S., Qian, L., Dong, X., ... & Xu, Z. (2020). A simulation training course for family medicine residents in China managing COVID-19. Australian Journal of General Practice, 49, 364–368. https://doi.org/10.31128/AJGP-04-20-5337
Shikino, K., Ito, S., Ohira, Y., Noda, K., Asahina, M., & Ikusaka, M. (2019). Usefulness of a short training seminar on how to handle difficult patients in simulated education. Advances in Medical Education and Practice, 10, 483–491. https://doi.org/10.2147/AMEP.S209573.
Shrestha, R., Badyal, D., Shrestha, A. P., & Shrestha, A. (2020). In-situ simulation-based module to train interns in resuscitation skills during cardiac arrest. Advances in Medical Education and Practice, 11, 271–285. https://doi.org/10.2147/AMEP.S246920.
Smith-Jentsch, K. A. (2018). How to conduct a high impact team self-evaluation session. Organizational Dynamics, 47, 107–114. https://doi.org/10.1016/j.orgdyn.2017.11.002
*Smith-Jentsch, K. A., Cannon-Bowers, J. A., Tannenbaum, S. I., & Salas, E. (2008). Guided team self-correction: Impacts on team mental models, processes, and effectiveness. Small Group Research, 39, 303–327. https://doi.org/10.1177/1046496408317794
*Soucisse, M. L., Boulva, K., Sideris, L., Drolet, P., Morin, M., & Dubé, P. (2017). Video coaching as an efficient teaching method for surgical residents—A randomized controlled trial. Journal of Surgical Education, 74, 365–371. https://doi.org/10.1016/j.jsurg.2016.09.002
Steinwachs, B. (1992). How to facilitate a debriefing. Simulation & Gaming, 23, 186–195. https://doi.org/10.1177/1046878192232006
Sundstrom, E., De Meuse, K. P., & Futrell, D. (1990). Work teams: Applications and effectiveness. American Psychologist, 45, 120–133. https://doi.org/10.1037/0003-066X.45.2.120
*Sutton, R. M., Niles, D., Meaney, P., Aplenc, R., French, B., Abella, B. S.,. . . Nadkami, V. (2009). Abstract P55: Brief bedside CPR skill debriefing and automated corrective feedback during simulated pediatric cardiac arrest. Circulation, 120, s1453.
Switzer, F. S., Paese, P. W., & Drasgow, F. (1992). Bootstrap estimates of standard errors in validity generalization. Journal of Applied Psychology, 77, 123–129. https://doi.org/10.1037/0021-9010.77.2.123
Tannenbaum, S. I., & Cerasoli, C. P. (2013). Do team and individual debriefs enhance performance? A meta-analysis. Human Factors, 55, 231–245. https://doi.org/10.1177/0018720812448394
Taylor, P. J., Russ-Eft, D. F., & Chan, D. W. (2005). A meta-analytic review of behavior modeling training. Journal of Applied Psychology, 90, 692–709. https://doi.org/10.1037/0021-9010.90.4.692
*Tudor, G. J., Podolej, G. S., Willemsen-Dunlap, A., Lau, V., Svendsen, J. D., McGarvey, J., ... & Barker, L. T. (2020). The equivalence of video self-review versus debriefing after simulation: Can faculty resources be reallocated? AEM Education and Training, 4, 36–42. https://doi.org/10.1002/aet2.10372.
U.S. Bureau of Labor Statistics. (2018). North American Industry Classification System (NAICS). Government Printing Office. Retrieved from https://www.bls.gov/bls/naics.htm.
Van Den Noortgate, W., & Onghena, P. (2005). Parametric and nonparametric bootstrap methods for meta-analysis. Behavior Research Methods, 37, 11–22. https://doi.org/10.3758/BF03206394
*Van Heukelom, J. N., Begaz, T., & Treat, R. (2010). Comparison of postsimulation debriefing versus in-simulation debriefing in medical simulation. Simulation in Healthcare, 5, 91–97. https://doi.org/10.1097/SIH.0b013e3181be0d17
*Vattanavanit, V., Khwannimit, B., & Nilmoje, T. (2020). Comparison of knowledge and confidence between medical students as leaders and followers in simulated resuscitation. International Journal of Medical Education, 11, 19–24. https://doi.org/10.5116/ijme.5e01.f00c
*Verkuyl, M., Atack, L., McCulloch, T., Liu, L., Betts, L., Lapum, J. L., . . . Romaniuk, D. (2018). Comparison of debriefing methods after a virtual simulation: An experiment. Clinical Simulation in Nursing, 19, 1–7. https://doi.org/10.1016/j.ecns.2018.03.002.
Villado, A. J. (2008). The after-action review training approach: An integrative framework and empirical investigation (Unpublished doctoral dissertation). Texas A&M University.
*Villado, A. J., & Arthur, W., Jr. (2013). The comparative effect of subjective and objective after-action reviews on team performance on a complex task. Journal of Applied Psychology, 98, 514–528. https://doi.org/10.1037/a0031510
Vroom, V. H. (1964). Work and motivation. Wiley.
Wachter, R., & Gupta, K. (2018). Understanding patient safety (3rd ed.). McGraw-Hill.
*Welke, T. M., LeBlanc, V. R., Savoldelli, G. L., Joo, H. S., Chandra, D. B., Crabtree, N. A., & Naik, V. N. (2009). Personalized oral debriefing versus standardized multimedia instruction after patient crisis simulation. Anesthesia and Analgesia, 109, 183–189. https://doi.org/10.1213/ane.0b013e3181a324ab
Wofford, J. C., Goodwin, V. L., & Premack, S. (1992). Meta-analysis of the antecedents of personal goal level and of the antecedents and consequences of goal commitment. Journal of Management, 18, 595–615. https://doi.org/10.1177/014920639201800309
Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes, 37, 60–82. https://doi.org/10.1016/0749-5978(86)90044-0
Wood, R. E., Mento, A. J., & Locke, E. A. (1987). Task complexity as a moderator of goal effects: A meta-analysis. Journal of Applied Psychology, 72, 416–425. https://doi.org/10.1037/0021-9010.72.3.416
*Zausig, Y. A., Grube, C., Boeker-Blum, T., Busch, C. J., Bayer, Y., Sinner, B., . . . Graf, B. M. (2009). Inefficacy of simulator-based training on anaesthesiologists’ non-technical skills. Acta Anaesthesiologica Scandinavica, 53, 611–619. https://doi.org/10.1111/j.1399-6576.2009.01946.x
*Zhang, H., Wang, W., Goh, S. H. L., Wu, X. V., & Mörelius, E. (2020). The impact of a three-phase video-assisted debriefing on nursing students' debriefing experiences, perceived stress and facilitators’ practices: A mixed methods study. Nurse Education Today, 90, 104460. https://doi.org/10.1016/j.nedt.2020.104460
Zigmont, J. J., Kappus, L. J., & Sudikoff, S. N. (2011). The 3D model of debriefing: Defusing, discovering, and deepening. Seminars in Perinatology, 35, 52–58. https://doi.org/10.1053/j.semperi.2011.01.003
Zou, G. Y. (2007). Toward using confidence intervals to compare correlations. Psychological Methods, 12, 399–413. https://doi.org/10.1037/1082-989X.12.4.399
Acknowledgements
We thank Anton Villado for his help with the double coding. We also thank Elle Keiser for her assistance with the coding of the healthcare tasks. Part of this research was performed while the first author held an NRC Research Associateship award at the Gaming Research Integration for Learning Laboratory (GRILL), under the Air Force Research Laboratory, 711th Human Performance Wing, Dayton, OH.
Disclaimer
The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Army, DOD, or the US Government.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
ESM 1
(DOCX 167 kb)
Rights and permissions
About this article
Cite this article
Keiser, N.L., Arthur, , W. A Meta-Analysis of Task and Training Characteristics that Contribute to or Attenuate the Effectiveness of the After-Action Review (or Debrief). J Bus Psychol 37, 953–976 (2022). https://doi.org/10.1007/s10869-021-09784-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10869-021-09784-x