Abstract
The development of extended reality in recent years is opening doors for using extended reality devices (virtual reality, augmented, and mixed reality devices) in education and healthcare. The purpose of this pilot study was to test the use of augmented reality in teaching healthcare practitioners in a stroke assessment simulation designed for clinical training, where students at nursing school are targets in the study. To conduct our feasibility, a simulation application was developed for the mixed reality device that projects a human face displaying facial drooping (a symptom of stroke) onto a computerized training mannequin. Nursing students were then placed in a clinical simulation wherein they wore the mixed reality device and performed an assessment of their mannequin patient to identify the symptom of stroke and act accordingly. The students completed a survey following their simulations, and then provided feedback on the device and the quality of their experience. The results of the study show that most students enjoyed the simulation and felt that extended reality would be a very useful educational tool for clinical training and healthcare. Further development of the program and device is underway, and future tests will be conducted. The results from this study will be helpful in further progressing the development of extended reality, and the use of these devices in healthcare training.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aebersold M (2011) Using simulation to improve the use of evidence-based practice guidelines. West J Nurs Res 33:296–305. https://doi.org/10.1177/0193945910379791
Aebersold M (2018) Simulation-based learning: no longer a novelty in undergraduate education. OJIN Online J Issues Nurs 23:15–20. https://doi.org/10.3912/OJIN.Vol23No02PPT39
Aebersold M, Kocan MJ, Tschannen D, Michaels J (2011) Use of simulation in stroke unit education. J Neurosci Nurs 43:349–353. https://doi.org/10.1097/JNN.0b013e318234e9ca
Aebersold M, Tschannen D, Stephens M et al (2012) Second Life®: a new strategy in educating nursing students. Clin Simul Nurs 8:e469–e475. https://doi.org/10.1016/j.ecns.2011.05.002
Aebersold M, Villarruel A, Tschannen D et al (2015) Using a virtual environment to deliver evidence-based interventions: the facilitator’s experience. JMIR Serious Games 3:e5. https://doi.org/10.2196/games.4293
Aebersold M, Voepel-Lewis T, Cherara L et al (2018) Interactive anatomy-augmented virtual simulation training. Clin Simul Nurs 15:34–41. https://doi.org/10.1016/j.ecns.2017.09.008
Akaike M, Fukutomi M, Nagamune M et al (2012) Simulation-based medical education in clinical skills laboratory. J Med Invest 59:28–35. https://doi.org/10.2152/jmi.59.28
Akula M, Lipman RR, Franaszek M et al (2013) Real-time drill monitoring and control using building information models augmented with 3D imaging data. Autom Constr 36:1–15. https://doi.org/10.1016/j.autcon.2013.08.010
American Heart Association (2018) Stroke warning signs and symptoms. https://www.strokeassociation.org/STROKEORG/WarningSigns/Stroke-Warning-Signs-and-Symptoms_UCM_308528_SubHomePage.jsp. Accessed 19 Jun 2018
Azuma RT (1997) A survey of augmented reality. Presence-Teleop Virt 6:355–385. https://doi.org/10.1162/pres.1997.6.4.355
Balian S, McGovern SK, Abella BS et al (2019) Feasibility of an augmented reality cardiopulmonary resuscitation training system for health care providers. Heliyon 5:e02205. https://doi.org/10.1016/j.heliyon.2019.e02205
Behzadan AH, Kamat VR (2006) Animation of construction activities in outdoor augmented reality. In: Proceedings of the joint international conference on computing and decision making in civil and building engineering (ICCCBE). Montréal, Canada, pp 1135–1143
Behzadan AH, Kamat VR (2009) Interactive augmented reality visualization for improved damage prevention and maintenance of underground infrastructure. In: Proceedings of the construction research congress (CRC). ASCE, Seattle, WA, USA, pp 1214–1222
Behzadan AH, Kamat VR (2011) Integrated information modeling and visual simulation of engineering operations using dynamic augmented reality scene graphs. J Inf Technol Constr (ITcon) 16:259–278
Behzadan AH, Kamat VR (2012) A framework for utilizing context-aware augmented reality visualization in engineering education. In: Proceedings of the international conference on construction applications of virtual reality (CONVR). Taipei, Taiwan, pp 292–299
Behzadan AH, Iqbal A, Kamat VR (2011) A collaborative augmented reality based modeling environment for construction engineering and management education. In: Proceedings of the winter simulation conference (WSC). IEEE, Phoenix, AZ, USA, pp 3568–3576
Behzadan AH, Dong S, Kamat VR (2015) Augmented reality visualization: a review of civil infrastructure system applications. Adv Eng Inform 29:252–267. https://doi.org/10.1016/j.aei.2015.03.005
Benjamin EJ, Muntner P, Alonso A et al (2019) Heart disease and stroke statistics—2019 update: a report from the american heart association. Circulation 139:e56–e528. https://doi.org/10.1161/CIR.0000000000000659
Bunch ME, Nunziato EC, Labovitz DL (2012) Barriers to the use of intravenous tissue plasminogen activator for in-hospital strokes. J Stroke Cerebrovasc 21:808–811. https://doi.org/10.1016/j.jstrokecerebrovasdis.2011.04.012
Cavallini A, Micieli G, Marcheselli S, Quaglini S (2003) Role of monitoring in management of acute ischemic stroke patients. Stroke 34:2599–2603. https://doi.org/10.1161/01.STR.0000094423.34841.BB
Caylor S, Aebersold M, Lapham J, Carlson E (2015) The use of virtual simulation and a modified TeamSTEPPSTM training for multiprofessional education. Clin Simul Nurs 11:163–171. https://doi.org/10.1016/j.ecns.2014.12.003
Chalhoub J, Ayer SK (2018) Using Mixed Reality for electrical construction design communication. Autom Constr 86:1–10. https://doi.org/10.1016/j.autcon.2017.10.028
Chang A, Llinas EJ, Chen K et al (2018) Shorter intensive care unit stays? The majority of post-intravenous tPA (tissue-type plasminogen activator) symptomatic hemorrhages occur within 12 hours of treatment. Stroke 49:1521–1524. https://doi.org/10.1161/STROKEAHA.118.021398
Cooper JB, Taqueti VR (2004) A brief history of the development of mannequin simulators for clinical education and training. BMJ Qual Saf 13:i11–i18. https://doi.org/10.1136/qshc.2004.009886
Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart 13:319–340. https://doi.org/10.2307/249008
de Freitas S, Neumann T (2009) The use of ‘exploratory learning’ for supporting immersive learning in virtual environments. Comput Educ 52:343–352. https://doi.org/10.1016/j.compedu.2008.09.010
de Ribaupierre S, Kapralos B, Haji F et al (2014) Healthcare training enhancement through virtual reality and serious games. In: Ma M, Jain LC, Anderson P (eds) Virtual, augmented reality and serious games for healthcare 1. Springer, Berlin, Heidelberg, Berlin, Heidelberg, pp 9–27
Dev P, Heinrichs WL (2008) Learning medicine through collaboration and action: collaborative, experiential, networked learning environments. Virtual Real 12:215–234. https://doi.org/10.1007/s10055-008-0099-5
Dong S, Behzadan AH, Feng C, Kamat VR (2013) Collaborative visualization of engineering processes using tabletop augmented reality. Adv Eng Softw 55:45–55. https://doi.org/10.1016/j.advengsoft.2012.09.001
Dulli D, Samaniego EA (2007) Inpatient and community ischemic strokes in a university hospital. Neuroepidemiology 28:86–92. https://doi.org/10.1159/000098551
Dunston PS, Arns LL, Mcglothlin JD et al (2011) An immersive virtual reality mock-up for design review of hospital patient rooms. In: Wang X, Tsai JJ-H (eds) Collaborative design in virtual environments. Springer, Dordrecht, pp 167–176
Fast-Berglund Å, Gong L, Li D (2018) Testing and validating extended reality (xR) technologies in manufacturing. Proc Manuf 25:31–38. https://doi.org/10.1016/j.promfg.2018.06.054
Feng C, Kamat VR (2013) Plane registration leveraged by global constraints for context-aware AEC applications. Comput-Aided Civ Inf 28:325–343. https://doi.org/10.1111/j.1467-8667.2012.00795.x
Freeland TR, Pathak S, Garrett RR et al (2016) Using medical mannequins to train nurses in stroke swallowing screening. Dysphagia 31:104–110. https://doi.org/10.1007/s00455-015-9666-6
Fussman C, Rafferty AP, Lyon-Callo S et al (2010) Lack of association between stroke symptom knowledge and intent to call 911. Stroke 41:1501–1507. https://doi.org/10.1161/STROKEAHA.110.578195
Harbison J, Hossain O, Jenkinson D et al (2003) Diagnostic accuracy of stroke referrals from primary care, emergency room physicians, and ambulance staff using the face arm speech test. Stroke 34:71–76. https://doi.org/10.1161/01.STR.0000044170.46643.5E
Harrington CM, Kavanagh DO, Quinlan JF et al (2018) Development and evaluation of a trauma decision-making simulator in Oculus virtual reality. Am J Surg 215:42–47. https://doi.org/10.1016/j.amjsurg.2017.02.011
Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge, UK, Second
HTC (2019) VIVETM | discover virtual reality beyond imagination. https://www.vive.com/eu/. Accessed 8 Mar 2019
Hughes CE, Stapleton CB, Hughes DE, Smith EM (2005) Mixed reality in education, entertainment, and training. IEEE Comput Graph 25:24–30. https://doi.org/10.1109/MCG.2005.139
Hung W-H, Liu C-W, Liang C-J, Kang S-C (2016) Strategies to accelerate the computation of erection paths for construction cranes. Automat Constr 62:1–13. https://doi.org/10.1016/j.autcon.2015.10.008
Jauch EC, Saver JL, Adams HP et al (2013) Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the american heart association/american stroke association. Stroke 44:870–947. https://doi.org/10.1161/STR.0b013e318284056a
Kamat VR, Behzadan AH (2006) GPS and 3DOF tracking for georeferenced registration of construction graphics in outdoor augmented reality. In: Proceedings of the EG-ICE international workshop on intelligent computing in engineering and architecture. Springer, Berlin, Heidelberg, Ascona, Switzerland, pp 368–375
Kawulich BB, D’Alba A (2018) Teaching qualitative research methods with second Life, a 3-dimensional online virtual environment. Virtual Real. https://doi.org/10.1007/s10055-018-0353-4
Kim J, You S, Lee S, et al (2015) Evaluation of human robot collaboration in masonry work using immersive virtual environments. In: Proceedings of the international conference on construction applications of virtual reality (CONVR). Banff, Canada, pp 132–141
Kobayashi L, Zhang XC, Collins SA et al (2018) Exploratory application of augmented reality/mixed reality devices for acute care procedure training. West J Emerg Med 19:158–164. https://doi.org/10.5811/westjem.2017.10.35026
Köhrmann M, Schellinger PD, Breuer L et al (2011) Avoiding in hospital delays and eliminating the three-hour effect in thrombolysis for stroke. Int J Stroke 6:493–497. https://doi.org/10.1111/j.1747-4949.2011.00585.x
Laerdal Medical (2019) SimMan 3G. https://www.laerdal.com/us/products/simulation-training/emergency-care-trauma/simman-3g/. Accessed 10 Mar 2019
Linden Lab (2018) Second Life—virtual worlds, virtual reality, VR, avatars, free 3D chat. https://secondlife.com/. Accessed 15 Jun 2018
Liang C-J, Kamat VR, Menassa CC (2018a) Real-time construction site layout and equipment monitoring. In: Proceedings of the construction research congress. New Orleans, LA, pp 64–74
Liang C-J, Lundeen KM, McGee W, et al (2018b) Stacked hourglass networks for markerless pose estimation of articulated construction robots. In: Proceedings of the international symposium on automation and robotics in construction (ISARC). IAARC, Berlin, Germany, pp 859–865
Liang C-J, Lundeen KM, McGee W et al (2019) A vision-based marker-less pose estimation system for articulated construction robots. Autom Constr 104:80–94. https://doi.org/10.1016/j.autcon.2019.04.004
Liaw Y-L, Lin K-Y, Li M, Chi N-W (2012) Learning assessment strategies for an educational construction safety video game. In: Proceedings of the construction research congress (CRC). ASCE, West Lafayette, IN, USA, pp 2091–2100
Mann S, Furness T, Yuan Y, et al (2018) All reality: virtual, augmented, mixed (x), mediated (x,y), and multimediated reality. arXiv:1804.08386 [cs]
Matsumoto K, Temiz Y, Taghavi H, et al (2019) An earbud-type wearable (A hearable) with vital parameter sensors for early detection and prevention of heat-stroke. In: Proceedings of the annual international conference of the IEEE engineering in medicine and biology society (EMBC). pp 7049–7055
Microsoft (2020) Microsoft Hololens | mixed reality technology for business. https://www.microsoft.com/da-dk/hololens. Accessed 27 Feb 2020
Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE T Inf Syst E77-D:1321–1329
Moskowitz MA, Lo EH, Iadecola C (2010) The science of stroke: mechanisms in search of treatments. Neuron 67:181–198. https://doi.org/10.1016/j.neuron.2010.07.002
Oculus (2019) Oculus Rift. https://www.oculus.com/rift/#oui-csl-rift-games=robo-recall. Accessed 8 Mar 2019
Olson E (2011) AprilTag: a robust and flexible visual fiducial system. In: Proceedings of the IEEE international conference on robotics and automation (ICRA). IEEE, Shanghai, China, pp 3400–3407
Palmarini R, Erkoyuncu JA, Roy R, Torabmostaedi H (2018) A systematic review of augmented reality applications in maintenance. Robot Cim-Int Manuf 49:215–228. https://doi.org/10.1016/j.rcim.2017.06.002
Pan Z, Cheok AD, Yang H et al (2006) Virtual reality and mixed reality for virtual learning environments. Comput Graph 30:20–28. https://doi.org/10.1016/j.cag.2005.10.004
Radu I (2014) Augmented reality in education: a meta-review and cross-media analysis. Pers Ubiquit Comput 18:1533–1543. https://doi.org/10.1007/s00779-013-0747-y
Rochlen LR, Levine R, Tait AR (2017) First-person point-of-view–augmented reality for central line insertion training: a usability and feasibility study. Simul Healthc J Soc Simul Healthc 12:57–62. https://doi.org/10.1097/SIH.0000000000000185
Roots A, Thomas L, Jaye P, Birns J (2011) Simulation training for hyperacute stroke unit nurses. Br J Nurs 20:1352–1356. https://doi.org/10.12968/bjon.2011.20.21.1352
Roquer J, Rodríguez-Campello A, Gomis M et al (2008) Acute stroke unit care and early neurological deterioration in ischemic stroke. J Neurol 255:1012–1017. https://doi.org/10.1007/s00415-008-0820-z
Samsung (2019) Samsung Gear VR with controller. In: The official Samsung Galaxy site. https://www.samsung.com/global/galaxy/gear-vr/. Accessed 8 Mar 2019
Saposnik G, Kapral MK, Coutts SB et al (2009) Do all age groups benefit from organized inpatient stroke care? Stroke 40:3321–3327. https://doi.org/10.1161/STROKEAHA.109.554907
Schwamm LH, Rosenthal ES, Hirshberg A et al (2004) Virtual TeleStroke support for the emergency department evaluation of acute stroke. Acad Emerg Med 11:1193–1197. https://doi.org/10.1197/j.aem.2004.08.014
Shirer M, Mainelli T, Ubrani J (2018) Demand for augmented reality/virtual reality headsets expected to rebound in 2018, says IDC. In: IDC: the premier global market intelligence company. https://www.idc.com/getdoc.jsp?containerId=prUS43639318. Accessed 17 Jun 2018
Singular Inversions (2019) FaceGen—3D Human Faces. https://facegen.com/. Accessed 10 Mar 2019
Talmaki S, Kamat VR (2014) Real-time hybrid virtuality for prevention of excavation related utility strikes. J Comput Civil Eng 28:04014001. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000269
Teng C-C, Jensen N, Smith T, et al (2018) Interactive augmented live virtual reality streaming: a health care application. In: Proceedings of the international conference on medical and health informatics. ACM, Tsukuba, Japan, pp 143–147
Teng C-C, Redfearn B, Nuttall C, et al (2019) Mixed reality patients monitoring application for critical care nurses. In: Proceedings of the international conference on medical and health informatics. ACM, Xiamen, China, pp 49–53
Tran L, Tran P, Tran L (2020) A cross-sectional analysis of 2017 stroke symptoms recognition at the US regional level. Chronic Illn. https://doi.org/10.1177/1742395320905650
Weiner E, McNew R, Trangenstein P, Gordon J (2010) Using the virtual reality world of second life to teach nursing faculty simulation management. In: Proceedings of the world congress on medical informatics. Cape Town, South Africa, pp 615–619
Wu T-H, Wu F, Liang C-J et al (2017) A virtual reality tool for training in global engineering collaboration. Univ Access Inf Soc. https://doi.org/10.1007/s10209-017-0594-0
Wüller H, Behrens J, Garthaus M et al (2019) A scoping review of augmented reality in nursing. BMC Nurs 18:19. https://doi.org/10.1186/s12912-019-0342-2
Yamal J-M, Rajan SS, Parker SA et al (2018) Benefits of stroke treatment delivered using a mobile stroke unit trial. Int J Stroke 13:321–327. https://doi.org/10.1177/1747493017711950
Yang SJ, Franco T, Wallace N et al (2019) Effectiveness of an interdisciplinary, nurse driven in-hospital code stroke protocol on in-patient ischemic stroke recognition and management. J Stroke Cerebrovasc Dis 28:104398. https://doi.org/10.1016/j.jstrokecerebrovasdis.2019.104398
Yoon SS, Heller RF, Levi C et al (2001) Knowledge of stroke risk factors, warning symptoms, and treatment among an australian urban population. Stroke 32:1926–1930. https://doi.org/10.1161/01.STR.32.8.1926
You S, Kim J-H, Lee S et al (2018) Enhancing perceived safety in human–robot collaborative construction using immersive virtual environments. Autom Constr 96:161–170. https://doi.org/10.1016/j.autcon.2018.09.008
Zhu E, Hadadgar A, Masiello I, Zary N (2014) Augmented reality in healthcare education: an integrative review. PeerJ 2:e469. https://doi.org/10.7717/peerj.469
Acknowledgements
The authors would like to acknowledge Microsoft Corporation for providing the MR headset HoloLens to our work, and the Business Engagement Center at the University of Michigan for facilitating our collaboration across the industry and academia. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of Microsoft Corporation.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liang, CJ., Start, C., Boley, H. et al. Enhancing stroke assessment simulation experience in clinical training using augmented reality. Virtual Reality 25, 575–584 (2021). https://doi.org/10.1007/s10055-020-00475-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-020-00475-1