Skip to main content
SpringerLink
Log in

My hands? Importance of personalised virtual hands in a neurorehabilitation scenario

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

We have developed a novel and affordable way to texture virtual hands from individually taken photographs and integrated the virtual hands into a mixed reality neurorehabilitation system. This mixed reality system allows for serious game play with mirrored and non-mirrored hands, designed for patients with unilateral motor impairments. Before we can ethically have patients use the system, we must show that embodiment can be achieved for healthy users. We compare our approach’s results to previous work in the field and present a study with 48 healthy (non-clinical) participants targeting visual fidelity and self-location. We show that embodiment can be achieved for mirrored and non-mirrored hand representations and that the higher realism of virtual hands achieved by our texturing approach alters perceived embodiment. We further evaluate whether using virtual hands resized to the individual’s hand size affects embodiment. We present a 16-participant study where we could not find a significant difference with personal resized hands. In addition to rehabilitation contexts, our findings have implications for the design and development of applications where embodiment is of high importance, such as surgical training and remote collaboration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Al-Tairi ZH, Rahmat RW, Saripan MI, Sulaiman PS (2014) Skin segmentation using yuv and rgb color spaces. J Inf Process Syst 10(2):283–299. https://doi.org/10.3745/JIPS.02.0002

    Article  Google Scholar 

  • Argelaguet F, Hoyet L, Trico M, Lécuyer A (2016) The role of interaction in virtual embodiment: effects of the virtual hand representation. In: Virtual reality (VR), 2016 IEEE. IEEE, pp 3–10

  • Assis GAd, Corrêa AGD, Martins MBR, Pedrozo WG, Lopes RdD (2016) An augmented reality system for upper-limb post-stroke motor rehabilitation: a feasibility study. Disabil Rehab Assist Technol 11(6):521–528

    Google Scholar 

  • Banakou D, Groten R, Slater M (2013) Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes. Proc Nat Acad Sci 110(31):12846–12851

    Article  Google Scholar 

  • Birckhead B, Khalil C, Liu X, Conovitz S, Rizzo A, Danovitch I, Bullock K, Spiegel B (2019) Recommendations for methodology of virtual reality clinical trials in health care by an international working group: Iterative study. JMIR Mental Health 6(1):e11973

    Article  Google Scholar 

  • Botvinick M, Cohen J (1998) Rubber hands’ feel’touch that eyes see. Nature 391(6669):756

    Article  Google Scholar 

  • Doidge N (2008) The brain that changes itself: stories of personal triumph from the frontiers of brain science. carlton north. Victoria: Scribe Publications

  • Freina L, Ott M (2015) A literature review on immersive virtual reality in education: state of the art and perspectives. In: The international scientific conference eLearning and software for education, vol 1, pp 10–1007

  • Giraux P, Sirigu A (2003) Illusory movements of the paralyzed limb restore motor cortex activity. Neuroimage 20:S107–S111

    Article  Google Scholar 

  • Gonzalez-Franco M, Perez-Marcos D, Spanlang B, Slater M (2010) The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment. In: Virtual reality conference (VR), 2010 IEEE. IEEE, pp 111–114

  • Hoermann S, Ferreira dos Santos L, Morkisch N, Jettkowski K, Sillis M, Devan H, Kanagasabai PS, Schmidt H, Krüger J, Dohle C et al (2017) Computerised mirror therapy with augmented reflection technology for early stroke rehabilitation: clinical feasibility and integration as an adjunct therapy. Disabil Rehabil 39(15):1503–1514

    Article  Google Scholar 

  • Holden MK (2005) Virtual environments for motor rehabilitation. Cyberpsychol Behav 8(3):187–211

    Article  Google Scholar 

  • Holmes D, Charles D, Morrow P, McClean S, McDonough S (2016) Usability and performance of leap motion and oculus rift for upper arm virtual reality stroke rehabilitation. In: Proceedings of the 11th international conference on disability, virtual reality & associated technologies, Central Archive at the University of Reading

  • Hung YX, Huang PC, Chen KT, Chu WC (2016) What do stroke patients look for in game-based rehabilitation: a survey study. Medicine 95(11):e3032

    Article  Google Scholar 

  • IJsselsteijn W, de Kort Y, Haans A (2005) Is this my hand i see before me? The rubber hand illusion in reality, virtual reality, and mixed reality 41–47

  • Iosa M, Morone G, Fusco A, Castagnoli M, Fusco FR, Pratesi L, Paolucci S (2015) Leap motion controlled videogame-based therapy for rehabilitation of elderly patients with subacute stroke: a feasibility pilot study. Top Stroke Rehabil 22(4):306–316

    Article  Google Scholar 

  • Jung S, Bruder G, Wisniewski PJ, Sandor C, Hughes CE (2018) Over my hand: Using a personalized hand in vr to improve object size estimation, body ownership, and presence. In: Proceedings of the symposium on spatial user interaction. ACM, pp 60–68

  • Khademi M, Mousavi Hondori H, McKenzie A, Dodakian L, Lopes CV, Cramer SC (2014) Free-hand interaction with leap motion controller for stroke rehabilitation. In: Proceedings of the extended abstracts of the 32nd annual ACM conference on human factors in computing systems. ACM, pp 1663–1668

  • Khan H, Lee G, Hoermann S, Clifford R, Billinghurst M, Lindeman R (2017) Evaluating the effects of hand-gesture-based interaction with virtual content in a 360 movie. In: ICAT-EGVE

  • Kilteni K, Groten R, Slater M (2012) The sense of embodiment in virtual reality. Presence Teleoper Virtual Environ 21(4):373–387

    Article  Google Scholar 

  • Lin L, Jörg S (2016) Need a hand? How appearance affects the virtual hand illusion. In: Proceedings of the ACM symposium on applied perception. ACM, pp 69–76

  • Liu Z, Zhang Y, Rau PLP, Choe P, Gulrez T (2015) Leap-motion based online interactive system for hand rehabilitation. In: International conference on cross-cultural design. Springer, pp 338–347

  • Ma K, Hommel B (2013) The virtual-hand illusion: effects of impact and threat on perceived ownership and affective resonance. Front Psychol 4:604

    Article  Google Scholar 

  • Ma K, Hommel B (2015) The role of agency for perceived ownership in the virtual hand illusion. Conscious Cogn 36:277–288

    Article  Google Scholar 

  • MacNeil LB (2017) Virtual mirror therapy system for stroke and acquired brain injury patients with hemiplegia. PhD thesis

  • Mueller F, Davis M, Bernard F, Sotnychenko O, Verschoor M, Otaduy MA, Casas D, Theobalt C (2019) Real-time pose and shape reconstruction of two interacting hands with a single depth camera. ACM Trans Graph 38(4):1–13

    Article  Google Scholar 

  • Nimcharoen C, Zollmann S, Collins J, Regenbrecht H (2018) Is that me?–embodiment and body perception with an augmented reality mirror. In: 2018 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct). IEEE, pp 158–163

  • Piryankova IV, Wong HY, Linkenauger SA, Stinson C, Longo MR, Bülthoff HH, Mohler BJ (2014) Owning an overweight or underweight body: distinguishing the physical, experienced and virtual body. PLoS ONE 9(8):e103428

    Article  Google Scholar 

  • Powers MB, Emmelkamp PM (2008) Virtual reality exposure therapy for anxiety disorders: A meta-analysis. J Anxiety Disord 22(3):561–569

    Article  Google Scholar 

  • Preston C, Kuper-Smith BJ, Ehrsson HH (2015) Owning the body in the mirror: the effect of visual perspective and mirror view on the full-body illusion. Sci Rep 5(1):1–10

    Article  Google Scholar 

  • Proença JP, Quaresma C, Vieira P (2018) Serious games for upper limb rehabilitation: a systematic review. Disabil Rehabil Assist Technol 13(1):95–100

    Article  Google Scholar 

  • Regenbrecht H, McGregor G, Ott C, Hoermann S, Schubert T, Hale L, Hoermann J, Dixon B, Franz E (2011) Out of reach? A novel ar interface approach for motor rehabilitation. In: 2011 10th IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 219–228

  • Romero J, Tzionas D, Black MJ (2017) Embodied hands: modeling and capturing hands and bodies together. ACM Trans Graph 36(6):245

    Article  Google Scholar 

  • Rother C, Kolmogorov V, Blake A (2004) “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314. https://doi.org/10.1145/1015706.1015720

    Article  Google Scholar 

  • Rubio B, Nirme J, Duarte E, Cuxart A, Rodriguez S, Duff A, Verschure P (2013) Virtual reality based tool for motor function assessment in stroke survivors. In: Converging clinical and engineering research on neurorehabilitation. Springer, pp 1037–1041

  • Schwind V, Knierim P, Tasci C, Franczak P, Haas N, Henze N (2017) These are not my hands!: Effect of gender on the perception of avatar hands in virtual reality. In: Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, pp 1577–1582

  • Sharp T, Keskin C, Robertson D, Taylor J, Shotton J, Kim D, Rhemann C, Leichter I, Vinnikov A, Wei Y, et al. (2015) Accurate, robust, and flexible real-time hand tracking. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, pp 3633–3642

  • Slater M, Perez-Marcos D, Ehrsson HH, Sanchez-Vives MV (2009) Inducing illusory ownership of a virtual body. Front Neurosci 3(2):214

    Article  Google Scholar 

  • Suzuki S, Abe K (1985) Topological structural analysis of digitized binary images by border following. Comput Vis Graph Image Process 30(1):32–46. https://doi.org/10.1016/0734-189X(85)90016-7

    Article  MATH  Google Scholar 

  • Trojan J, Diers M, Fuchs X, Bach F, Bekrater-Bodmann R, Foell J, Kamping S, Rance M, Maaß H, Flor H (2014) An augmented reality home-training system based on the mirror training and imagery approach. Behav Res Methods 46(3):634–640

    Article  Google Scholar 

  • Tung JY, Lulic T, Gonzalez DA, Tran J, Dickerson CR, Roy EA (2015) Evaluation of a portable markerless finger position capture device: accuracy of the leap motion controller in healthy adults. Physiol Meas 36(5):1025

    Article  Google Scholar 

  • Weber LM, Nilsen DM, Gillen G, Yoon J, Stein J (2019) Immersive virtual reality mirror therapy for upper limb recovery after stroke: a pilot study. Am J Phys Med Rehabil 98(9):783–788

    Article  Google Scholar 

  • Yuan Y, Steed A (2010) Is the rubber hand illusion induced by immersive virtual reality? In: 2010 IEEE virtual reality conference (VR). IEEE, pp 95–102

Download references

Author information

Authors and Affiliations

  1. Department of Information Science, University of Otago, Dunedin, New Zealand

    Chris Heinrich, Matthew Cook, Tobias Langlotz & Holger Regenbrecht

Authors
  1. Chris Heinrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tobias Langlotz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Holger Regenbrecht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chris Heinrich.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 75602 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Heinrich, C., Cook, M., Langlotz, T. et al. My hands? Importance of personalised virtual hands in a neurorehabilitation scenario. Virtual Reality 25, 313–330 (2021). https://doi.org/10.1007/s10055-020-00456-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-020-00456-4

Keywords

Search

Navigation