Abstract
In the perception of self-motion, visual cues originating from an embodied humanoid avatar seen from a first-person perspective (1st-PP) are processed in the same way as those originating from a person’s own body. Here, we sought to determine whether the user’s and avatar’s bodies in virtual reality have to be colocalized for this visual integration. In Experiment 1, participants saw a whole-body avatar in a virtual mirror facing them. The mirror perspective could be supplemented with a fully visible 1st-PP avatar or a suggested one (with the arms hidden by a virtual board). In Experiment 2, the avatar was viewed from the mirror perspective or a third-person perspective (3rd-PP) rotated 90° left or right. During an initial embodiment phase in both experiments, the avatar’s forearms faithfully reproduced the participant’s real movements. Next, kinaesthetic illusions were induced on the static right arm from the vision of passive displacements of the avatar’s arms enhanced by passive displacement of the participant’s left arm. Results showed that this manipulation elicited kinaesthetic illusions regardless of the avatar’s perspective in Experiments 1 and 2. However, illusions were more likely to occur when the mirror perspective was supplemented with the view of the 1st-PP avatar’s body than with the mirror perspective only (Experiment 1), just as they are more likely to occur in the latter condition than with the 3rd-PP (Experiment 2). Our results show that colocalization of the user’s and avatar’s bodies is an important, but not essential, factor in visual integration for self-motion perception.
Similar content being viewed by others
Kinaesthesia is the conscious perception of one’s own movements. It emerges from the integration of muscular, tactile, and visual signals from the individual’s body (Blanchard et al., 2013; Chancel, Brun, et al., 2016; Romano et al., 2013) and cannot be considered independently of the perception of one’s own body, also termed “embodiment” (Arzy et al., 2006). However, the representation of one’s body is a rather flexible construct and can stretch to incorporate external objects like fake body segments (e.g., a rubber hand; Botvinick & Cohen, 1998; Chancel & Ehrsson, 2020; Ehrsson et al., 2005; Longo et al., 2008) and virtual whole bodies (e.g., avatars; Pfeiffer et al., 2013; Slater et al., 2010). As proposed by Kilteni and collaborators (2012) and in line with de Vignemont (2011), “the sense of embodiment toward an artificial body B is the sense that emerges when B’s properties are processed as if they were the properties of one’s own biological body” (p. 375). This inevitably raises the question whether or not these “external” objects or bodies are treated as part of one’s own body for the purpose of kinaesthesia.
Recent research has shown that visual motion signals originating from either an embodied rubber hand (Metral & Guerraz, 2019) or an embodied whole-body avatar (Giroux et al., 2019; Giroux et al., 2018) undergo multisensory integration for the purpose of kinaesthesia in the same way as signals from the biological body do (see Barra et al., 2020, for a review)—at least when these external objects are viewed from a first-person perspective (1st-PP). Indeed, displacement of an embodied humanoid 1st-PP avatar’s forearm evoked (in most individuals) an illusory movement in their otherwise static, real forearm (Giroux et al., 2019; Giroux et al., 2018). The visual inputs from the avatar’s arms were combined with congruent muscle inputs from one of the participant’s arms moved passively; these illusory movements were far more vivid than those evoked with visual or muscular unimodal stimulation (Giroux et al., 2019; Giroux et al., 2018). This finding is in line with the multisensory integration framework and therefore appears to be a relevant way of highlighting the involvement of visual motion cues in kinaesthesia (see also Chancel, Blanchard, et al., 2016; Chancel et al., 2017; Hagura et al., 2007). From a 1st-PP, the user’s body and its avatar are colocalized. The latter is seen as occupying the space where the user’s body should be, according to the user’s view of the virtual environment. Colocalization of the avatar with the user’s body facilitates embodiment, as embodiment is more limited when avatars are viewed from a third-person perspective (3rd-PP; Bertamini et al., 2011; Jenkinson & Preston, 2015; Kontaris & Downing, 2011; Maselli & Slater, 2014; Preston et al., 2015). The 1st-PP can even override the 3rd-PP view of one’s real body, leading to an out-of-body illusion and disembodiment of the body viewed from a 3rd-PP (Ehrsson, 2007; Guterstam et al., 2015; Guterstam & Ehrsson, 2012). When facing a mirror, the reflection of someone’s body is also not colocalized with the real body but is located “on the other side” of the mirror. Nevertheless, humans learn to recognize themselves in the mirror at a very young age (Anderson, 1984) and to use the mirror reflection to guide their movements when (for example) putting on makeup or combing their hair (Bertamini & Parks, 2005; Bianchi et al., 2008; Lawson & Bertamini, 2006). Indeed, most interactions with mirrors are about recognizing and perceiving oneself—giving mirrors a special role in the perception of one’s own body. Furthermore, a rubber hand or a whole-body avatar viewed in a mirror can be embodied in the same way as an avatar viewed from a 1st-PP (Abdulkarim & Ehrsson 2016; Bertamini et al., 2011; González-Franco et al., 2010; Jenkinson & Preston, 2015; Kontaris & Downing, 2011). This finding shows that the avatar real (physical) position and orientation are reconstructed from its reflection in the mirror and that it is processed as being part of one’s own body (Bertamini et al., 2011; Jenkinson & Preston, 2015; Kontaris & Downing, 2011). We therefore hypothesized that the mirror perspective could provide relevant visual motion cues about one’s own body for the purpose of kinaesthesia. In a first experiment (Experiment 1), we immersed our participants in a virtual environment in which a whole-body avatar was visible in a virtual mirror facing them. This “mirror perspective” was evaluated either with or without an additional 1st-PP. The 1st-PP avatar was itself either fully visible or had its arms hidden by a virtual board (i.e., they were only suggested). The involvement of visual motion cues in kinaesthesia was assessed by measuring the characteristics of illusory arm displacements evoked by manipulation of the avatar’s arm movements (see Giroux et al., 2019; Giroux et al., 2018). In line with the abovementioned hypothesis, we expected illusory displacements to occur when the avatar was viewed in the mirror perspective (i.e., even in the absence of a 1st-PP). However, in view of the rules of multisensory integration in general and the impact of the richness and reliability of visual signals on perception in particular (Chancel, Blanchard, et al., 2016; Ernst & Banks, 2002), we expected the illusions to be more frequent and more intense when either the 1st-PP avatar is added to the mirror perspective or is only suggested making the mirror perspective more reliable.
It is known that a 3rd-PP is less effective for avatar embodiment than a 1st-PP or a mirror perspective (Denisova & Cairns, 2015; Maselli & Slater, 2013; Petkova et al., 2011; Slater et al., 2010), as spatial congruence may be an important element for multisensory integration (Calvert & Thesen, 2004; Lopez et al., 2012; Slutsky & Recanzone, 2001; Tsakiris & Haggard, 2005). In fact, embodiment also named the sense of embodiment (Kilteni et al., 2012), consists of an ensemble of subcomponents (ownership, self-location, agency; Kilteni et al., 2012; Longo et al., 2008; and external appearance; González-Franco & Peck, 2018). Therefore, the effect of the avatar perspective may differ from one subcomponent to another. Indeed, “body ownership” (i.e., the feeling that the avatar’s body is one’s own), “self-location” (i.e., the feeling that one’s own body is located in the same place as the avatar’s body), and “external appearance” (i.e., the feeling that one resembles the avatar) may be found to be weaker (though still possibly present) with a 3rd-PP than with a 1st-PP (Debarba et al., 2017; Gorisse et al., 2017). In contrast, “agency” (i.e., the feeling that one can control the avatar’s body as though it was its own) can be equally strong with 3rd-PP and 1st-PP—particularly when the avatar faithfully reproduces the participant’s actual movements (Gorisse et al., 2017), as manipulated in the embodiment phase in the present experiment. Thus, a 3rd-PP avatar in virtual reality can—at least to some extent—be felt as if it were one’s own body. Further, this perspective can be used effectively to act in the virtual environment as well as in real, more or less remote environments (i.e., teleoperation). In this respect, one can reasonably expect visual motion cues originating from a 3rd-PP avatar to provide relevant visual motion cues for the perception of one’s own movements, though to a lesser extent than the mirror perspective, for instance. Experiment 2 was designed to test the latter hypothesis. We immersed our participants in a virtual environment in which their whole-body avatar was viewed from a mirror perspective (facing them, although in the absence of a virtual mirror) or from a 3rd-PP rotated 90° to the left or right.
Methods
Participants
The expected effect size for Experiment 1 was based on that observed in Giroux et al.’s (2018) study of the involvement of visual motion cues from an embodied 1st-PP avatar on kinaesthetic illusions (n = 28; d = 0.92). An a priori power analysis with α = 0.05 and 1 − β = 0.80 indicated a minimum required sample size of n ≈ 12 (RStudio software, RStudio Team, 2020). Considering the comparison of interest in the present study (i.e., a context of spatial translation thought to lead to smaller effects), we doubled the minimum required sample size to n ≈ 24. The same number was chosen for Experiment 2. Our sample sizes were preregistered before conducting the experiments (see Open Practices Statement).
The avatar used in both experiments was female; hence, only female participants were recruited. Some individuals do not experience any visually induced kinaesthetic illusions (Chancel et al., 2017; Giroux et al., 2019; Giroux et al., 2018; Guerraz et al., 2012). We therefore screened the participants in a preliminary test assessing kinaesthetic illusion with combined visual and muscular stimulation, as performed by Giroux et al. (2018) and Giroux et al. (2019). Of the 26 and 28 healthy participants respectively recruited for Experiments 1 and 2, 24 and 26 experienced the kinaesthetic illusion and were therefore included. The mean ± SD age of the study population was 19.17 ± 0.96 years for Experiment 1 and 20.62 ± 6.64 years for Experiment 2. None of the participants took part in both experiments. All but six of the participants were right-handed (as determined in the Edinburgh Inventory Test; Oldfield, 1971). None of the participants had proprioceptive or neuromuscular disorders. All of the participants had normal or corrected-to-normal vision and provided their written, informed consent prior to initiation of the experiments. The study was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and was approved by the local independent ethics committee (CEREUS, Savoie Mont Blanc University, Chambéry, France).
Materials
Immersive virtual reality technology
Participants were seated in front of a desk on which two manipulanda were positioned. These consisted of wooden arms ending with a handgrip on which the participants placed their forearms. The right manipulandum remained static, while the left one, which was fitted with a low-noise synchronous DC motor (24V, Maxon with planetary Maxon reductor 1296:1 Switzerland), could flex or extend the participant’s arm from its initial starting position. A head-mounted display (HMD; Oculus Rift; Oculus VR, Irvine, CA, USA) operating with Unity software (Unity Technologies, San Francisco, CA, USA) and running on a computer equipped with an MSI Geforce GTX 980 Gaming 4G graphics card (Micro-Star International, Taiwan) and an Intel Core i7–4790K processor (Intel Corporation, USA) immersed the participants in a virtual environment in which they saw a young female avatar. The avatar was seated with its elbows on a virtual table and could be viewed from different perspectives, depending on the experimental condition. The virtual forearms could reproduce the same movements as the participant’s forearms, thanks to an electromagnetic motion capture system (Liberty™, Polhemus, Colchester, VT, USA). The motion sensors were fixed to wrist pads worn by the participants to prevent any wrist movements. The head-mounted display included real-time tracking mechanisms to estimate the head’s position and orientation. This feature enables perspective correction and allows the user to move his or her head freely in a room-sized environment, while the avatar’s head reproduces the same movements. Only the forearms’ movements and rotations and the tilting of the participant’s head were reproduced on the avatar in the virtual reality. The avatar’s other body parts remained static. White noise was played through the head-mounted display’s headphones to cover the sound of the manipulandum motor. A representation of the experimental setup is shown in Fig. 1.
The avatar’s perspectives in Experiment 1
In Experiment 1, a virtual mirror was placed in front of the participant. This showed the reflection of the 1st-PP avatar, with the avatar’s arms visible in the Avatar conditions (see Fig. 2a–c) or hidden from view by virtual white boxes in the Control conditions (Fig. 2d–f). The visual perspective in the Avatar conditions was manipulated as follows: In the Mirror_P condition (Fig. 2a), the avatar could be seen in the mirror, but not from the 1st-PP. In the Mirror+_P condition (Fig. 2b), the avatar could be seen in the mirror and from the 1st-PP, but with the forearms in 1st-PP masked by a virtual board and therefore only suggested (Fig. 2b). In the Full_P condition (Fig. 2c), the avatar was fully visible in the mirror and from the 1st-PP. A video for each of these experimental conditions can be viewed from the following address (https://doi.org/10.6084/m9.figshare.13359158).
The avatar’s perspectives in Experiment 2
In Experiment 2, the avatar was seated in front of the participant. It either faced the participant (the Mirror_P condition; Fig. 3a–d) or was turned 90° to the left or to the right (3rd_PP conditions; Fig. 3b, c, e, f). A 1st-PP avatar was not present in Experiment 2. A video for each of these experimental conditions can be viewed from the following address (https://doi.org/10.6084/m9.figshare.13359179).
Procedures
The avatar embodiment phase
Once the participant had been fitted with the experimental material as described in the previous section, she sat with her elbows on a table. She was required to look at the avatar’s arms in the virtual environment and to actively flex and extend her forearms (alternating phase and antiphase movements) at a comfortable, self-paced frequency for 1 minute. During this embodiment phase, the avatar’s forearms faithfully reproduced the participant’s real movements, as captured by the motion capture system. The spatiotemporal congruency between real, active movements and virtual movements enabled the participant to embody the avatar (see Dummer et al., 2009; Tsakiris et al., 2005). In Experiments 1 and 2, the reflected forearms and the 3rd-PP avatar’s forearms moved in a mirror-like way during the embodiment phases. The participant’s left arm controlled the avatar’s right arm, and vice versa. The embodiment phase was repeated for each perspective condition in a counterbalanced order and was immediately followed by a brief embodiment questionnaire. To maintain embodiment in a given perspective condition, an additional embodiment period of 30 seconds (not followed by the questionnaire) was performed between the blocks of six trials. The questionnaire contained items from González-Franco and Peck’s (2018) embodiment questionnaire. Fifteen items of relevance to our virtual reality design were used for Experiment 1 (1–5, 6–9, 14–15, 17–20), and 13 were used for Experiment 2 (1-3, 6–9, 14–15, 17–20). Items 4 and 5 (related to virtual mirrors) were only relevant in Experiment 1. Each assertion (e.g., “I felt as if the virtual arms were my own arms”) is scored on a 7-point Likert scale, from −3 (strongly disagree) to 3 (strongly agree). A mean embodiment score was computed for each participant, according to González-Franco and Peck’s recommendations. Four embodiment subscores were calculated from the various items: body ownership (Items 1–5), agency (Items 6–9), self-location (Items 14–15), and external appearance (Items 17–20). The items are presented in the Supplementary Material.
The illusion induction phase
After the embodiment phase, the participant’s forearms were positioned on the manipulanda. The right arm remained static and was held at 30° to the horizontal plane. The left arm was positioned at either 15° or 45° relative to the horizontal plane. Following a baseline period of a few seconds, the left arm could be passively moved with an angular amplitude of 30° and a constant angular speed of 3.8°/s (duration: 8 s). The sequence of movements consisted of six alternating flexions and extensions (six trials), with the first movement being either a flexion or an extension (counterbalanced between the participants). The participants were told not to resist these passive movements and to look at the avatar’s forearms. Each movement was followed by a few seconds during which the avatar’s and participant’s arms remained static, allowing the participant to verbally grade the speed of the illusory displacement experienced in the right arm on a 0-to-10 scale. A rating of 0 corresponded to the absence of illusory displacement, and 10 corresponded to the same speed of displacement as for the passively moved left forearm. Next, the participants had to indicate the direction of the perceived movement, which could be either the same as or opposite to that of the left forearm. On the basis of this second evaluation, the 0-to-10 speed rating was considered to be either positive when the illusory movement was in the direction of the left forearm, or negative when it was in the opposite direction. Hence, the final speed rating ranged from −10 to 10. The participant was also required to estimate the beginning and the end of the illusory displacement with respect to the start of the trial, with both estimates ranging from 0 s to 8 s (e.g., “it started three seconds after the beginning of the trial” was rated as 3; “it ended seven seconds after the beginning of the trial” was rated as 7). Once these variables had been rated, the next trial began.
In the Avatar conditions, passive displacement of the participant’s left forearm was coupled in real time to displacement of the left and right avatar’s forearms, both of which moved with the same angular amplitude and speed (from either 15° to 45° or from 45° to 15°, relative to the horizontal plane). In the Control conditions, the participant’s left forearm was moved passively, and the avatar’s arms were localized inside virtual boxes and were therefore not visible either in the mirror or from a 1st-PP or 3rd-PP.
Each experimental condition was performed in two blocks of six trials with three flexions and extensions per block. This led to a total of 36 experimental trials per participant for Experiment 1 as well as for Experiment 2. For each experimental condition, two familiarization trials were performed before the experimental block of six trials. The various experimental conditions were applied in a pseudorandom order. In Experiment 2, we randomly added two sham trials in each block. In a sham trial, the avatar’s forearms were masked in virtual white boxes and no displacement of the participant’s arm occurred. The sham trials were not included in the statistical analyses but allowed us to determine whether an illusion occurred in the absence of stimulation. Conceivably, such illusions could reflect either experimental demand or “spontaneous sensations” (see Beaudoin & Michael, 2014; Michael et al., 2015).
Statistical analyses
Analysis of variance and pairwise comparisons t tests were assessed with Bayesian equivalent tests. Statistical analysis was performed using JASP software (JASP Team, 2020 [https://jasp-stats.org/]; Kruschke 2010; Wagenmakers et al., 2018). Statistical evidence was reported using Bayes factors (BFs), BF10 for paired sample comparisons and BFincl for analyses of variance (ANOVAs) denoting the level of evidence of the alternate hypothesis (nonsigned difference), and the inclusion of a specific parameter in a model (ANOVA) respectively. The cutoff values defined by Jeffreys (1998) were used to interpret BFs.
Results
Experiment 1
The kinaesthetic illusion’s occurrence, speed, and duration were assessed in Experiment 1 in 3 × 2 within-subjects, repeated-measures Bayesian ANOVAs [“Perspective” (Mirror_P, Mirror+_P, Full_P) × “Avatar” (Avatar, Control)]. Embodiment and subcomponents scores were evaluated with a one-way repeated-measures Bayesian ANOVA [“Perspective” (Mirror_P, Mirror+_P, Full_P)]. Mean data are provided in Table 1.
The occurrence of the kinaesthetic illusion
The analysis of variance (ANOVA) yielded strong statistical evidence for an effect of Avatar (BFincl = 6.1e+15) and of Perspective (BFincl = 10.8), but no convincing interaction between Avatar and Perspective (BFincl = 1.51). As shown in Fig. 4 and confirmed by pairwise comparisons, the mean frequency of occurrence was greater when the avatar’s arms were visible (the Avatar condition) than when they were not (the Control condition), whatever the perspective (Mirror_P: BF10 = 745, CI95% [0.78, 2.03], Cohen’s d = 1.37; Mirror+_P: BF10 = 1.6e+3, CI95% [0.73, 2.15], d = 1.43; Full_P: BF10 = 5.3e+5, CI95% [1.53, 3.04], d = 2.47). In the Avatar condition, participants were also more likely to experience an illusion with the Full_P than with either the Mirror_P (BF10 = 37.2, CI95% [0.36, 1.58], d = 1.12) or Mirror+_P (BF10 = 44.9, CI95% [0.37, 1.62], d = 1.00). The latter two conditions did not differ from each other (BF10 = 0.34, CI95% [−0.77, 0.28], d = 0.21). There were no differences between the Control conditions (all BF10 < 1). It should be noted that the vast majority of the kinaesthetic illusions were in the same direction as the avatar’s forearm (and/or participant’s forearm) displacements. However, 1.9% and 7.2% of the illusions were reversed in the Avatar and Control conditions, respectively.
The speed and duration of the kinaesthetic illusion
The ANOVAs applied to both speed and duration showed evidence for effects of Avatar (speed: BFincl = 6.01e+15; duration: BFincl > 10e+15) and Perspective (speed: BFincl = 228.7; duration: BFincl = 1997.6) and an interaction between the two (speed: BFincl = 9.1; duration: BFincl = 122.6). As shown in Fig. 4, the mean speed was higher (Fig. 4b) and the mean duration was longer (Fig. 4c) in the Avatar condition than in the Control condition; this was confirmed by pairwise comparisons (all BF10 > 100). The mean speed and mean duration were also higher and longer, respectively, in the Avatar condition with Full_P than with Mirror+_P (speed: BF10 = 62.1, CI95% [0.42, 1.67], d = 0.87; duration: BF10 = 499, CI95% [0.63, 1.97], d = 0.97) or Mirror_P (speed: BF10 = 357, CI95% [0.62, 1.92], d = 0.87; duration: BF10 2515, CI95% [0.85, 2.92], d = 1.11). The latter two conditions did not differ from each other (speed: BF10 = 0.22, CI95% [−0.45, 0.57], d = 0.04; duration: BF10 = 0.28, CI95% [−0.32, 0.69], d = 0.14). In the Control condition, there was no evidence of a difference between Mirror_P, Mirror+_P, and Full_P (speed: BF10 < 1.5 for all; duration: BF10 < 0.5 for all).
The embodiment score
On a scale from −3 (no embodiment) to 3 (perfect embodiment), the mean ± standard deviation (SD) embodiment score was 0.86 (0.79), 0.92 (0.72), and 1.24 (0.53), in the Mirror_P, Mirror+_P and Full_P conditions, respectively. These scores were far from −3 (strongly disagree) and so probably reflected avatar embodiment in each Perspective condition. The ANOVA showed only anecdotal evidence of an effect of Perspective (BFincl = 2.02), with a slightly stronger embodiment for Full-P than for Mirror+_P (BF10 = 1.91, CI95% [0.03, 1.12], d = 0.51) or Mirror_P (BF10 = 2.13, CI95% [0.05, 1.15], d = 0.56). The latter two conditions did not differ from each other (BF10 = 0.23, CI95% [−0.42, 0.61], d = 0.08). Concerning the subscores, the ANOVA yielded substantial evidence for an effect of Perspective on Self-Location (BFincl = 3.24). Indeed, as shown in Fig. 5, participants felt that the avatar was more colocalized with their own body in the Full_P condition than in the Mirror_P condition (BF10 = 3.38, CI95% [0.09, 1.21], d = 0.75). There was no evidence of a difference between Full_P and Mirror+_P (BF10 = 0.74, CI95% [−0.02, 0.96], d = 0.54), and there was anecdotal evidence for the null hypothesis (H0) between Mirror_P and Mirror+_P (BF10 = 0.34, CI95% [−0.27, 0.77], d = 0.20). ANOVA analyses gave anecdotal or substantial evidence for H0 for Ownership (BFincl = 0.17), Agency (BFincl = 0.35) and Appearance (BFincl = 0.34).
Brief discussion of Experiment 1
As assessed through kinaesthetic illusion, visual motion cues also participate in kinaesthesia when they originate from an embodied avatar that is not colocalized with the participant’s body, but viewed from a mirror perspective. Indeed, kinaesthetic illusions were more frequent, faster, and longer-lasting when the movement was seen in the avatar’s arms in a mirror (the Avatar condition) than when they were not visible (the Control condition). The purpose of Experiment 2 was to test whether this result is specific to the mirror perspective, a perspective that is experienced in everyday life, or whether it can be extended to an avatar that is not colocalized with participant’s body, but also rotated 90° with respect to it.
Experiment 2
The kinaesthetic illusion’s occurrence, speed and duration were assessed in 2 × 2 within-subjects Bayesian ANOVA [“Perspective” (Mirror_P, 3rd_PP) × “Avatar” (Avatar, Control)]. Embodiment scores were evaluated with Bayesian pairwise comparisons t tests [“Perspective” (Mirror_P, 3rd_PP)]. Mean data are provided in Table 2.
The occurrence of the kinaesthetic illusion
The ANOVA yielded statistical evidence for an effect of Avatar (BFincl = 1.3e+8), an effect of Perspective (BFincl = 2.85), and an interaction between the two (BFincl = 8.52). As shown in Fig. 6a, participants were more likely to experience an illusion in the Avatar condition than in the Control condition, with both Mirror_P (BF10 = 1.21e+4, CI95% [0.98, 2.30], d = 1.48) and 3rd_PP (BF10 = 15.6, CI95% [0.23, 1.41], d = 0.71). Furthermore, the mean occurrence in the Avatar condition was higher for Mirror-P than for 3rd_PP (BF10 = 131, CI95% [0.47, 1.70], d = 0.58). No difference was observed in the Control condition (BF10 = 0.34, CI95% [−0.75, 0.28], d = 0.14). A few illusions occurred in the opposite direction to that of the avatar’s displacements (6% and 9.2% of the Avatar and Control trials, respectively). In the sham trials, a kinaesthetic illusion occurred in 10.6% of the cases.
The speed and duration of the kinaesthetic illusion
The ANOVAs of the speed and duration data showed evidence of an effect of Avatar (speed: BFincl = 7.5e+9; duration: BFincl = 1.22e+6), an effect of Perspective (speed: BFincl = 7.6; duration: BFincl = 1.89), and an interaction between the two (speed: BFincl = 6.8; duration: BFincl = 3.9). As shown in Fig. 6, the mean speed was higher (Fig. 6b), and the mean duration was longer (Fig. 6c) in the Avatar condition than in the Control condition, for both Mirror_P (speed: BF10 = 6.2e+4, CI95% [1.17, 2.52], d = 1.41; duration: BF10 = 585, CI95% [0.65, 1.88], d = 1.31) and 3rd_PP (speed: BF10 = 821, CI95% [0.68, 1.92], d = 1.02; duration: BF10 = 10.3, CI95% [0.23, 1.34], d = 0.75). The illusions were also faster and lasted longer in the Avatar condition with Mirror_P than with 3rd_PP (speed: BF10 = 36.7, CI95% [0.374, 1.52], d = 0.58; duration: BF10 = 10.9, CI95% [0.24, 1.35], d = 0.55); this was not the case in the Control condition (speed: BF10 = 0.22, CI95% [−0.42, 0.57], d = 0.07; duration: BF10 = 0.22, CI95% [−0.58, 0.41], d = 0.06).
The embodiment score
The mean (SD) embodiment score was 0.73 (0.59) and 0.48 (0.55) for Mirror_P and 3rd_PP, respectively. There was only anecdotal evidence for a higher embodiment score for Mirror_P than for 3rd_PP (BF10 = 1.55, CI95% [−0.002, 1.07], d = 0.44). Subcomponent analyses revealed that participants felt that the mirror avatar (Mirror_P) was more strongly colocalized with their own body than the 90°-rotated avatar (3rd_PP) (BF10 = 4.04, CI95% [0.11, 1.21], d = 0.40). Concerning the other three subscores, results showed only very limited evidence for H1 for Appearance (BF10 = 1.31, CI95% [−0.05, 1.04], d = 0.35) and substantial evidence for H0 for Ownership and Agency (BF10 < 0.3).
The Mirror_P condition in Experiment 1 and 2
The Mirror_P condition in Experiment 1 was the same as the Mirror_P condition in Experiment 2, except that the avatar was viewed in a virtual mirror frame in the former, but not in the latter. To check whether the presence of a virtual mirror affected kinaesthetic illusions and avatar embodiment, we used a Bayesian t test for independent samples to compare the Mirror_P condition of Experiment 1 with that of Experiment 2. The results showed that there was anecdotal to substantial evidence for H0 for the occurrence, speed, and duration of the kinaesthetic illusion and for the mean embodiment and subcomponent scores (all BF10 < 1). Therefore, viewing the avatar facing the participant, with or without a virtual mirror, did not affect kinaesthetic illusions nor embodiment.
Brief discussion of Experiment 2
Experiment 2 shows that visual motion cues can be used for kinaesthesia even when they originate from an embodied avatar that is seen from a third-person perspective and rotated 90°. Indeed, kinaesthetic illusions in one arm (combined by the actual move of the other arm) were more frequent and slightly stronger when both arms of the avatar were seen to move (the Avatar condition) than when they were not visible (the Control condition). However, these illusions were less frequent and weaker than those induced when the avatar was seen from a mirror perspective, indicating that the mirror perspective has a specific status among potential third-person perspectives.
Discussion
The objective of the present study was to establish whether visual motion cues from an embodied avatar that was not colocalized with the participant’s physical body contribute to kinaesthesia. This contribution was assessed by measuring the characteristics of illusory arm movements evoked by manipulation of the avatar’s arm movements. In Experiment 1, the avatar was viewed from a mirror perspective only (Mirror_P) or from a mirror perspective to which a fully visible or suggested 1st-PP avatar was added (Full_P and Mirror+_P, respectively). In Experiment 2, the avatar was viewed from a mirror perspective (Mirror_P) or from a 3rd-PP rotated 90° to the left or to the right (3rd_PP). Before the induction of any kinaesthetic illusions, participants were required to move their arms at a comfortable self-paced frequency. The avatar’s forearms faithfully reproduced these movements. Whatever the setup and the experimental conditions considered, 1 minute of this visuo-motor activity appeared to be enough to make participants embody their avatar (as evaluated with the embodiment questionnaire), albeit with slight differences between the various perspective conditions. In line with our starting hypotheses, the study’s results revealed that regardless of the perspective (Mirror_P, Mirror+_P, Full_P, 3rd_PP), kinaesthetic illusions in the hidden static right arm were more frequent, faster, and longer-lasting when displacements of the avatar’s arms were combined with passive movements of the left arm (the Avatar condition) than when the avatar’s arm were not visible (the Control condition). This finding indicates that visual motion cues from an avatar are used to perceive one’s own movements even when that avatar is not viewed from a 1st-PP. However, the characteristics of the illusions varied from one condition to the next. The illusions were most frequent and strongest when the avatar was fully visible in the mirror and from the 1st-PP (Full-P) and were least frequent and weakest when the avatar was viewed from a 3rd-PP only.
Use of visual motion cues from the mirror perspective and 3rd-PP for kinaesthesia
A key finding in Experiment 1 was that displacements of the avatar’s arms viewed from a mirror perspective, combined with passive movements of the left arm (the Avatar condition) induced stronger kinaesthetic illusions in the static (right) forearm than when the avatar’s arms were hidden from view (the Control condition). Therefore, visual motion cues from the avatar viewed in a virtual mirror facing the participant (self-avatar) contributed to kinaesthesia, and therefore corroborated Jenkinson and Preston’s (2015) statement that arm movements viewed in a mirror are processed as one’s own movements and not as an external object’s. An additional finding of Experiment 1 was that induced illusions were more frequent and stronger in Full_P (when a 1st-PP view of the avatar was combined with the mirror perspective) than in Mirror_P or Mirror+_P (a mirror perspective only). This result was not unexpected, given that we have shown repeatedly that vision of an embodied avatar from a 1st-PP alone can induce kinaesthetic illusions (Giroux et al., 2018; Giroux et al., 2019). However, the mean illusion speed (7.16/10) and mean duration (6.49 s) for Full-P in Experiment 1 were greater than or at least similar to those observed in our previous studies with the 1st-PP only (Giroux et al., 2018: mean speed: 6.30/10; mean duration: 5.56 s; Giroux et al., 2019: mean speed: 5.98/10; mean duration: 5.01 s).
The main finding in Experiment 2 was that the combination of displacement of the arms of a 90°-rotated 3rd-PP embodied avatar with the passive displacement of the participant’s left arm induced more frequent, stronger kinaesthetic illusions in the static (right) forearm than when the avatar’s arms were hidden from view (the control condition). This finding indicates that visual motion cues from a 90°-rotated 3rd-PP avatar can be used for kinaesthesia. However, although the degree of the illusions experienced with the 3rd-PP avatar was strongly related to those experienced with the mirror avatar (speed: r =.70, p < .01; duration: r = .63, p < .01), they were weaker than them.
Taken as a whole, our results show that the impact of visual cues on kinaesthesia increases in proportion to the available amount of concordant and coherent visual information about self-movement. This may be the consequence of different integration processes. In the well-known maximum likelihood estimation (MLE) framework proposed by Ernst and Banks (2002; see also Ernst & Bülthoff, 2004), multisensory integration results from the weighted average of the different sensory modalities involved, the weights of which being proportional to the relative reliability of each sensorial modality. The lack of congruence between the external motion signals with the 3rd_PP avatar and the participant’s position in space may limit the reliability of that signal and therefore its weight in the final percept. In addition, integration of redundant sensory cues does not occur only between, but also within senses (van Dam et al., 2014). In that respect, the superiority of the Full_P to induce kinaesthetic illusion as compared with the Mirror_P in Experiment 1, may result from the integration of visual motion cues from the 1st-PP avatar with that from the avatar seen through the mirror, increasing consequently the overall weight of visual cues in the final percept. The involvement of these different processes in the genesis of the present kinaesthetic illusions remains, however, speculative, as the experimental setup of Experiment 1 and 2 were not specifically designed to test it.
It is worth mentioning that the illusion reported here is some sort of virtual reality replica of the well-known physical “mirror illusion.” In the physical mirror illusion, participants move one arm in a mirror positioned in the sagittal plane and, thanks to the left–right body symmetry, perceive the reflected arm (or hand) as their own, and mistakenly experience a bimanual movement of both arms, although only one arm is moving. This is similar in the present experiments, as the two arms could be represented in the virtual environment and only one is moving. We repeatedly showed that this illusion of bimanual movement (either the physical mirror illusion or its virtual reality replica), often considered as a prototypic visual illusion involves nonvisual signals (bilateral proprioceptive-somaesthetic signals, in fact; see Hakuta et al., 2014; Izumizaki et al., 2010) that interact with the visual signals and strengthen the kinaesthetic effect (Chancel, Brun, et al., 2016; Chancel et al., 2017; Giroux et al., 2018). This potentiation of visual motion signals by proprioceptive ones may partly explain why rather equivocal visual motion signals about the self, can be integrated for kinaesthesia, even when provided by a mirror facing the participant or from a third-person perspective 90° rotated. It may also account for the higher sensitivity of the illusion (in terms of occurrence and strength) to changes of visual perspective as compared with the sense of embodiment which appeared to be more stable over the different perspective conditions of Experiments 1 and 2 (see below for the discussion about the sense of embodiment). Taken all together, our results show the extreme plasticity of body representation since, for kinaesthesia, it is not mandatory that the visual information of the body segments stand in the reachable space but they can be presented in the extrapersonal space.
Cognitive processes and self-motion perception
At present, it is no longer necessary to demonstrate that perception (including the perception of one’s own body or body movement) does not solely result from bottom-up processing (i.e., a simple flow of afferent sensory information from the periphery to the cortex). In fact, cognitive processes often modulate the upstream processing of sensory stimuli. For example, visuo-tactile synchronization is not sufficient for self-attribution of an object because cognitive processing modulates the strength of self-attribution according to the anatomical and morphological similarities between the object and the internally stored representation of the body (de Vignemont & Farnè, 2010; Haans et al., 2008; Kilteni et al., 2012; Tsakiris et al., 2010; Tsakiris & Haggard, 2005; Waltemate et al., 2018). As is the case for body representation, the CNS relies on prior experience for self-motion perception. For instance, Dieter et al. (2014) reported that we can perceive “visual” movement in the complete absence of the primary visual input that ordinarily triggers that perceptual experience. Indeed, the execution of hand movements in a way that normally results in retina-driven visual experiences can be enough to generate the corresponding visual perceptions. Not to mention this extreme case, expectations, prior knowledge or beliefs about one’s body are involved in multisensory integration because they modulate the reliability of a particular item of sensory information. In the field of perception, this cognitive modulation is often referred to as Bayesian inference (Debats & Heuer, 2018; Kersten et al., 2004). The occurrence of kinaesthetic illusions with Mirror_P or 3rd_PP in Experiments 1 and 2 demonstrates that the movements of avatar’s arm are processed as one’s own movements and not as those of an external object—even though the avatar is not physically colocalized with the participant’s body. This is doubtless related to the ability (acquired at a very young age; Anderson, 1984) to recognize ourselves from an external point of view (e.g., in a mirror) and to use these external visual cues to guide our movements (Bertamini & Parks, 2005; Bianchi et al., 2008; Lawson & Bertamini, 2006). This ability attests to the involvement of cognitive processes in kinaesthesia.
Our results also showed that the presence of a “virtual mirror” was not a prerequisite for the occurrence of an illusion in the Mirror_P condition, since similar illusions were observed in Experiment 1 (with a virtual mirror) and in Experiment 2 (without a virtual mirror). It seems that as long as the avatar facing the participant is expected to move its arms according to the participant’s will (as experienced during the embodiment phase), it is perceived to reflect one’s own body regardless of whether or not it is framed by a mirror. Hence, the avatar’s arm movements are processed as being one’s own movements and not as those of an external (nonembodied) object. These results are somewhat surprising, given that how a given item of visual information is processed may depend on whether it is coming from a mirror or not. For example, a dark silhouette can activate different brain areas depending on whether it is imagined to be oneself viewed from a 3rd-PP or oneself viewed in a mirror (Arzy et al., 2006). In contrast to Arzy et al.’s (2006) findings, our participants’ movements were tightly coupled to those of the avatar during the embodiment phase. This strong visuomotor coupling and immersion in a virtual world might explain why the participants were able to imagine a mirror when the (virtual) mirror frame was absent.
We expected in Experiment 1 to observe more frequent, stronger illusions not only when the avatar’s 1st-PP was added to the mirror perspective (in Full_P) but also when it was suggested (as in Mirror+_P). We hypothesized that the provision of implicit cues related to the presence of a 1st-PP (as with transparent avatars; see Guterstam et al., 2015, Guterstam et al., 2013; Dieter et al., 2014) would reinforce the mirror avatar’s reliability as a reflection of one’s own body. This hypothesis was not confirmed by our results, as we did not observe any difference (substantial evidence for H0), whatever the variable considered, between the suggested 1st-PP avatar (Mirror+_P) and the lack of a 1st-PP avatar (Mirror_P). Therefore, the suggestion of an avatar’s 1-PP did not seem to enhance the reliability and therefore the weight attributed to visual motion cues from the mirror avatar in the kinaesthetic percept.
Embodiment of an avatar and visual perspectives
It has long been known (Tastevin, 1937) that healthy participants can perceive an artificial body segment as part of their own body when the latter is hidden from view but moved or stroked synchronously with the artificial body segment (for a review, see Tsakiris, 2010). Embodiment can even be extended to full bodies (Beaudoin et al., 2020; Giroux et al., 2018; Petkova & Ehrsson, 2008; Serino et al., 2013), as was observed in the present study. Our results showed that embodiment via synchronous visuomotor coupling does not necessarily require colocalization between the avatar and the participant’s body (1st-PP), since an avatar viewed from a mirror perspective and (to a lesser extent) an avatar viewed from a 3rd-PP were sufficient for embodiment. This indicates that spatial incongruence between the participant’s biological body and that of the avatar (90° rotation) constrains embodiment (Blanke et al., 2015; Calvert & Thesen, 2004; González-Franco & Peck, 2018, Slutsky & Recanzone, 2001; Tsakiris & Haggard, 2005), but does not prevent it. Mirror or 3rd-PP avatars may be processed in a very specific way and may become body extensions or even body substitutes, thus extending the peri-personal space as do mirror images (Maravita et al., 2002; Noel et al., 2015) or even tools (Berti & Frassinetti, 2000; Guerraz et al., 2018; Miller et al., 2018).
However, the embodiment scores reported in Experiments 1 and 2 must be interpreted with caution because of the absence of a control condition in which no embodiment is expected (with visuomotor asynchrony, for example). The absence of a control condition without visuomotor synchrony prevented us from measuring the full extent of embodiment, since cognitive biases or demand characteristics may lead to responses above the minimum score (−3: fully disagree) when evaluating each of the assertions in the embodiment questionnaire. However, our results are consistent with the large number of literature reports attesting to the embodiment of artificial body parts (Bertamini et al., 2011; Jenkinson & Preston, 2015; Kontaris & Downing, 2011; Preston et al., 2015) and whole virtual bodies (Slater et al., 2010; Debarba et al., 2017) especially following visuomotor coupling. For example, Debarba et al. (2017) showed that illusory ownership (a subcomponent of embodiment) of a virtual body and response to threats can be achieved not only with a 1st-PP but also with a 3rd-PP under congruent visuomotor-tactile conditions—although the latter were stronger for the 1st-PP than for the 3rd-PP (see also Gorisse et al., 2017).
The sense of embodiment consists of different subcomponents—namely ownership, self-location, agency (Kilteni et al., 2012), and external appearance (González-Franco & Peck, 2018), which may be affected to different extents by the avatar’s perspective. Our results confirmed this differential effect of perspective, since only “self-location” varied from one perspective condition to another. As suggested by Kilteni et al. (2012), “self-location is a determinate volume in space where one feels to be located” (p. 375 ). Self-location and body-space usually coincide, though in the context of virtual reality this colocalization can artificially be broken down according to the visual perspective. Unsurprisingly, our results showed that the sensation that one’s own body is localized at the same place as the avatar’s body was stronger in the Full_P condition, in which participants can see their virtual body from a first-person perspective, than under the mirror perspective conditions in Experiment 1, just as it was stronger under the mirror perspective condition than in the 3rd_PP in Experiment 2.
In contrast to self-location, agency, body ownership, but also external appearance, were similar in the various perspective conditions of Experiments 1 and 2. This was expected concerning agency. As defined by Blanke and Metzinger (2009), agency refers to having a “global motor control, including the subjective experience of action, control, intention, motor selection and the conscious experience of will” (p. 9). In the context of virtual reality, agency refers therefore to the feeling that one can control the avatar’s body as though it was one’s own. The lack of differences between the different perspective conditions in Experiments 1 and 2 (also reported by Gorisse et al., 2017) is most likely the consequence of the visuomotor coupling (self-generated movement and congruent visual feedback) experienced during the experiments’ embodiment phase of Experiments 1 and 2.
Visuomotor coupling makes a person feel that he or she fully controls the avatar and may also change the way he or she consciously experience his bodies (Kalckert & Ehrsson, 2012; Ma & Hommel, 2015). The visuomotor coupling experienced by our participants in the embodiment phase of Experiments 1 and 2 may have therefore enhanced the feeling that the avatar was the source of the experienced sensations (body ownership), as reported previously by several authors (Dummer et al., 2009; Shimada et al., 2009; Tsakiris et al., 2005), and masked any potential effect of perspective. This interpretation must, however, be taken with caution, as other have not found conclusive evidence for such a relationship between agency and ownership and consider that the two components are related to different, independent psychological processes (Kalckert & Ehrsson, 2012).
In future research, it will be important to establish the causal relationship between embodiment and kinaesthetic illusions. As shown by the results of Experiments 1 and 2, embodiment (particularly the “self-location” subcomponent) and kinaesthetic illusions seem to change in the same direction; both were stronger in Full_P than in Mirror_P and stronger in Mirror_P than in 3rd_PP. As mentioned above, our experimental design prevented us from investigating this relationship. To do so, one would have to compare experimental scenarios in which embodiment is expected to be strong (such as with visuomotor synchrony as manipulated in Experiments 1 and 2) with those in which it is expected to be weak or even absent (such as with visuomotor asynchrony; for instance, see Dummer et al., 2009; Kalckert & Ehrsson, 2012; Ma & Hommel, 2015; Tsakiris et al., 2005). Comparison of visually induced kinaesthetic illusions under these conditions would highlight the relationship between embodiment (and each of its subcomponents) and kinaesthesia.
Conclusion
Our results confirmed that the visual perspective influences the weight allocated to visual-motion cues originating from an embodied avatar in the kinaesthetic percept. However, it appears that colocalization between the user’s body and the avatar’s body (viewed with a 1st-PP) is not a prerequisite for visual integration. These results are particularly relevant in the context of professional or recreational activities using virtual reality.
References
Abdulkarim, Z., & Ehrsson H. H. (2016). No causal link between changes in hand position sense and feeling of limb ownership in the rubber hand illusion. Attention, Perception, & Psychophysics, 78(2), 707–720. https://doi.org/10.3758/s13414-015-1016-0
Anderson, J. R. (1984). The development of self-recognition: A review. Developmental Psychobiology, 17(1), 35–49. https://doi.org/10.1002/dev.420170104
Arzy, S., Overney, L. S., Landis, T., & Blanke, O. (2006). Neural mechanisms of embodiment: Asomatognosia due to premotor cortex damage. Archives of Neurology, 63, 1022–1025. https://doi.org/10.1001/archneur.63.7.1022
Barra, J., Giroux, M., Metral, M., Cian, C., Luyat, M., Kavounoudias, A., & Guerraz, M. (2020). Functional properties of extended body representations in the context of kinesthesia. Neurophysiologie Clinique/Clinical Neurophysiology, 50, 455–465. https://doi.org/10.1016/j.neucli.2020.10.011
Beaudoin, M., Barra, J., Dupraz, L., Mollier-Sabet, P., & Guerraz, M. (2020). The impact of embodying an “elderly” body avatar on motor imagery. Experimental Brain Research, 238(6), 1467–1478. https://doi.org/10.1007/s00221-020-05828-5
Beaudoin, R., & Michael, G. A. (2014). Gating of spontaneous somatic sensations by movement. Somatosensory & Motor Research, 31(3), 111–121. https://doi.org/10.3109/08990220.2014.888992
Bertamini, M., Berselli, N., Bode, C., Lawson, R., & Wong, L. T. (2011). The rubber hand illusion in a mirror. Consciousness and Cognition, 20(4), 1108–1119. https://doi.org/10.1016/j.concog.2011.04.006
Bertamini, M., & Parks, T. E. (2005). On what people know about images on mirrors. Cognition, 98(1), 85–104. https://doi.org/10.1016/j.cognition.2004.11.002
Berti, A., & Frassinetti, F. (2000). When far becomes near: Remapping of space by tool use. Journal of Cognitive Neuroscience, 12(3), 415–420. https://doi.org/10.1162/089892900562237
Bianchi, I., Savardi, U., & Bertamini, M. (2008). Estimation and representation of head size (people overestimate the size of their head—Evidence starting from the 15th century). British Journal of Psychology, 99(4), 513–531. https://doi.org/10.1348/000712608X304469
Blanchard, C., Roll, R., Roll, J.-P., & Kavounoudias, A. (2013). Differential contributions of vision, touch and muscle proprioception to the coding of hand movements. PLOS ONE, 8(4), Article e62475. https://doi.org/10.1371/journal.pone.0062475
Blanke, O., & Metzinger, T. (2009). Full-body illusions and minimal phenomenal selfhood. Trends in Cognitive Science, 13(1), 7–13. https://doi.org/10.1016/j.tics.2008.10.003
Blanke, O., Slater, M., & Serino, A. (2015). Behavioral, neural, and computational principles of bodily self-consciousness. Neuron, 88, 145–166. https://doi.org/10.1016/j.neuron.2015.09.029
Botvinick, M., & Cohen, J. (1998). Rubber hands’ feel’ touch that eyes see. Nature, 391, Article 756. https://doi.org/10.1038/35784
Calvert, G. A., & Thesen, T. (2004). Multisensory integration: Methodological approaches and emerging principles in the human brain. Journal of Physiology–Paris, 98(1), 191–205. https://doi.org/10.1016/j.jphysparis.2004.03.018
Chancel, M., Blanchard, C., Guerraz, M., Montagnini, A., & Kavounoudias, A. (2016). Optimal visuotactile integration for velocity discrimination of self-hand movements. Journal of Neurophysiology, 116(3), 1522–1535. https://doi.org/10.1152/jn.00883.2015
Chancel, M., Brun, C., Kavounoudias, A., & Guerraz, M. (2016). The kinaesthetic mirror illusion: How much does the mirror matter? Experimental Brain Research, 234(6), 1459–1468. https://doi.org/10.1007/s00221-015-4549-5
Chancel, M., Kavounoudias, A., & Guerraz, M. (2017). What’s left of the mirror illusion when the mirror can no longer be seen? Bilateral integration of proprioceptive afferents! Neuroscience, 362, 118–126. https://doi.org/10.1016/j.neuroscience.2017.08.036
Chancel, M., & Ehrsson, H.H. (2020). Which hand is mine? Discriminating body ownership perception in a two-alternative forced-choice task. Attention, Perception & Psychophysics, 82(8), 4058–4083. https://doi.org/10.3758/s13414-020-02107
Debarba, H. G., Bovet, S., Salomon, R., Blanke, O., Herbelin, B., & Boulic, R. (2017). Characterizing first and third person viewpoints and their alternation for embodied interaction in virtual reality. PLOS ONE, 12(12), e0190109. https://doi.org/10.1371/journal.pone.0190109
Debats, N. B., & Heuer, H. (2018). Optimal integration of actions and their visual effects is based on both online and prior causality evidence. Scientific Reports, 8(1), Article 9796. https://doi.org/10.1038/s41598-018-28251-x
Denisova, A., & Cairns, P. (2015). First person vs. third person perspective in digital games: Do player preferences affect immersion? Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 145–148). https://doi.org/10.1145/2702123.2702256
de Vignemont, F. (2011). Embodiment, ownership and disownership. Consciousness and Cognition, 20(1), 82–93. https://doi.org/10.1016/j.concog.2010.09.004
de Vignemont, F. & Farné, A. (2010). Widening the body to rubber hands and tools: What’s the difference? Revue de neuropsychologie, 2(3), 203–211. https://doi.org/10.1684/nrp.2010.0087
Dieter, K. C., Hu, B., Knill, D. C., Blake, R., & Tadin, D. (2014). Kinesthesis can make an invisible hand visible. Psychological Science, 25, 66–75. https://doi.org/10.1177/0956797613497968
Dummer, T., Picot-Annand, A., Neal, T., & Moore, C. (2009). Movement and the rubber hand illusion. Perception, 38(2), 271–280. https://doi.org/10.1068/p5921
Ehrsson, H. H. (2007). The experimental induction of out-of-body experiences. Science, 317(5841), 1048–1048. https://doi.org/10.1126/science.1142175
Ehrsson, H. H., Holmes, N. T., & Passingham, R. E. (2005). Touching a rubber hand: Feeling of body ownership is associated with activity in multisensory brain areas. Journal of Neuroscience, 25(45), 10564–10573. https://doi.org/10.1523/JNEUROSCI.0800-05.2005
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429–433. https://doi.org/10.1038/415429a
Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8(4), 162–169. https://doi.org/10.1016/j.tics.2004.02.002
Giroux, M., Barra, J., Barraud, P.-A., Graff, C., & Guerraz, M. (2019). From embodiment of a point-light display in virtual reality to perception of one’s own movements. Neuroscience, 416, 30–40. https://doi.org/10.1016/j.neuroscience.2019.07.043
Giroux, M., Barra, J., Zrelli, I.-E., Barraud, P.-A., Cian, C., & Guerraz, M. (2018). The respective contributions of visual and proprioceptive afferents to the mirror illusion in virtual reality. PLOS ONE, 13(8), Article e0203086. https://doi.org/10.1371/journal.pone.0203086
González-Franco, M., & Peck, T. C. (2018). Avatar embodiment: Towards a standardized questionnaire. Frontiers in Robotics and AI, 5. https://doi.org/10.3389/frobt.2018.00074
González-Franco, M., Pérez-Marcos, D., Spanlang B., & Slater, M. (2010). The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment. Proceedings of the IEEE Virtual Reality Conference (pp. 111–114). https://doi.org/10.1109/VR.2010.5444805
Gorisse, G., Christmann, O., Amato, E. A., & Richir, S. (2017). First- and third-person perspectives in immersive virtual environments: Presence and performance analysis of embodied users. Frontiers in Robotics and AI, 4. https://doi.org/10.3389/frobt.2017.00033
Guerraz, M., Breen, A., Pollidoro, L., Luyat, M., & Kavounoudias, A. (2018). Contribution of visual motion cues from a held tool to kinesthesia. Neuroscience, 388, 11–22. https://doi.org/10.1016/j.neuroscience.2018.06.048
Guerraz, M., Provost, S., Narison, R., Brugnon, A., Virolle, S., & Bresciani, J.-P. (2012). Integration of visual and proprioceptive afferents in kinesthesia. Neuroscience, 223, 258–268. https://doi.org/10.1016/j.neuroscience.2012.07.059
Guterstam, A., Abdulkarim, Z., & Ehrsson, H. H. (2015). Illusory ownership of an invisible body reduces autonomic and subjective social anxiety responses. Scientific Reports, 5, Article 9831. https://doi.org/10.1038/srep09831
Guterstam, A., & Ehrsson, H. H. (2012). Disowning one’s seen real body during an out-of-body illusion. Consciousness and Cognition, 21(2), 1037–1042. https://doi.org/10.1016/j.concog.2012.01.018
Guterstam, A., Gentile, G., & Ehrsson, H.H. (2013). The invisible hand illusion: Multisensory integration leads to the embodiment of a discrete volume of empty space. Journal of Cognitive Neuroscience, 25, 1078–1099. https://doi.org/10.1162/jocn_a_00393
Hagura, N., Takei, T., Hirose, S., Aramaki, Y., Matsumura, M., Sadato, N., & Naito, E. (2007). Activity in the posterior parietal cortex mediates visual dominance over kinesthesia. Journal of Neuroscience, 27, 7047–7053.
Hakuta, N., Izumizaki, M., Kigawa, K., Murai, N., Atsumi, T., & Homma, I. (2014). Proprioceptive illusions created by vibration of one arm are altered by vibrating the other arm. Experimental Brain Research, 232, 2197–2206.
Haans, A., IJsselsteijn, W. A., & de Kort, Y. A. W. (2008). The effect of similarities in skin texture and hand shape on perceived ownership of a fake limb. Body Image, 5(4), 389–394. https://doi.org/10.1016/j.bodyim.2008.04.003
Izumizaki, M., Tsuge, M., Akai, L., Proske, U., & Homma, I. (2010). The illusion of changed position and movement from vibrating one arm is altered by vision or movement of the other arm. Journal of Physiology, 588, 2789–2800. https://doi.org/10.1113/jphysiol.2010.192336
JASP Team (2020). JASP (Version 0.14.1)[Computer software].
Jenkinson, P. M., & Preston, C. (2015). New reflections on agency and body ownership: The moving rubber hand illusion in the mirror. Consciousness and Cognition, 33, 432–442. https://doi.org/10.1016/j.concog.2015.02.020
Jeffreys, H. (1998). The theory of probability. Oxford University Press.
Kalckert, A., & Ehrsson, H.H. (2012). Moving a rubber hand that feels like your own: A dissociation of ownership and agency. Frontiers in Human Neuroscience, 6, Article 40. https://doi.org/10.3389/fnhum.2012.00040
Kersten, D., Mamassian, P., & Yuille, A. (2004). Object perception as Bayesian inference. Annual Review of Psychology, 55, 271–304. https://doi.org/10.1146/annurev.psych.55.090902.142005
Kilteni, K., Groten, R., & Slater, M. (2012). The sense of embodiment in virtual reality. Presence: Teleoperators and Virtual Environments, 21(4), 373–387. https://doi.org/10.1162/PRES_a_00124
Kontaris, I., & Downing, P. E. (2011). Reflections on the hand: The use of a mirror highlights the contributions of interpreted and retinotopic representations in the rubber-hand illusion. Perception, 40(11), 1320–1334. https://doi.org/10.1068/p6825
Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Science, 14, 293–300. https://doi.org/10.1016/j.tics.2010.05.001
Lawson, R., & Bertamini, M. (2006). Errors in judging information about reflections in mirrors. Perception, 35(9), 1265–1288. https://doi.org/10.1068/p5498
Longo, M., Schüür, F., Kammers, M., Tsakiris, M., & Haggard, P. (2008). What is embodiment? A psychometric approach. Cognition, 107(3), 978–998. https://doi.org/10.1016/j.cognition.2007.12.004
Lopez, C., Bieri, C. P., Preuss, N., & Mast, F. W. (2012) Tactile and vestibular mechanisms underlying ownership for body parts: a non-visual variant of the rubber hand illusion. Neuroscience Letters, 511(2), 120–124. https://doi.org/10.1016/j.neulet.2012.01.055
Ma, K., & Hommel, B. (2015). The role of agency for perceived ownership in the virtual hand illusion. Consciousness & Cognition, 36, 277–288. https://doi.org/10.1016/j.concog.2015.07.008
Maravita, A., Spence, C., Sergent, C., & Driver, J. (2002). Seeing your own touched hands in a mirror modulates cross-modal interactions. Psychological Science, 13(4), 350–355. https://doi.org/10.1111/j.0956-7976.2002.00463.x
Maselli, A., & Slater, M. (2013). The building blocks of the full body ownership illusion. Frontiers in Human Neuroscience, 7. https://doi.org/10.3389/fnhum.2013.00083
Maselli, A., & Slater, M. (2014). Sliding perspectives: Dissociating ownership from self-location during full body illusions in virtual reality. Frontiers in Human Neuroscience, 8. https://doi.org/10.3389/fnhum.2014.00693
Metral, M., & Guerraz, M. (2019). The fake hand in movement: visual motion cues from the rubber hand are processed for the purpose of kinesthesia. Consciousness & Cognition, 73, Article 102761. https://doi.org/10.1016/j.concog.2019.05.009.
Michael, G. A., Naveteur, J., Dupuy, M.-A., & Jacquot, L. (2015). My heart is in my hands: The interoceptive nature of the spontaneous sensations felt on the hands. Physiology & Behavior, 143, 113–120. https://doi.org/10.1016/j.physbeh.2015.02.030
Miller, L. E., Montroni, L., Koun, E., Salemme, R., Hayward, & V., Farnè, A. (2018). Sensing with tools extends somatosensory processing beyond the body. Nature, 561, 239–242. https://doi.org/10.1038/s41586-018-0460-0
Noel, J.-P., Pfeiffer, C., Blanke, O., & Serino, A. (2015). Peripersonal space as the space of the bodily self. Cognition, 144, 49–57. https://doi.org/10.1016/j.cognition.2015.07.012
Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh Inventory. Neuropsychologia, 9(1), 97–113. https://doi.org/10.1016/0028-3932(71)90067-4
Petkova, V. I., & Ehrsson, H. H. (2008). If I were you: Perceptual illusion of body swapping. PLOS ONE, 3(12), Article e3832. https://doi.org/10.1371/journal.pone.0003832
Petkova, V. I., Khoshnevis, M., & Ehrsson, H. H. (2011). The perspective matters! Multisensory Integration in ego-centric reference frames determines full-body ownership. Frontiers in Psychology, 2. https://doi.org/10.3389/fpsyg.2011.00035
Pfeiffer, C., Lopez, C., Schmutz, V., Duenas, J. A., Martuzzi, R., & Blanke, O. (2013). Multisensory origin of the subjective first-person perspective: Visual, tactile, and vestibular mechanisms. PLOS ONE, 8(4), Article e61751. https://doi.org/10.1371/journal.pone.0061751
Preston, C., Kuper-Smith, B. J., & Ehrsson, H. H. (2015). Owning the body in the mirror: The effect of visual perspective and mirror view on the full-body illusion. Scientific Reports, 5, Article 18345. https://doi.org/10.1038/srep18345
Romano, D., Bottini, G., & Maravita, A. (2013). Perceptual effects of the mirror box training in normal subjects. Restorative Neurology and Neuroscience, 31(4), 373–386. https://doi.org/10.3233/RNN-120273
RStudio Team (2020). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA. http://www.rstudio.com/
Serino, A., Alsmith, A., Costantini, M., Mandrigin, A., Tajadura-Jimenez, A., & Lopez, C. (2013). Bodily ownership and self-location: Components of bodily self-consciousness. Consciousness & Cognition, 22(4), 1239–1252. https://doi.org/10.1016/j.concog.2013.08.013
Shimada, S., Qi, Y., & Hiraki, K. (2009). Detection of visual feedback delay in active and passive self-body movements. Experimental. Brain Research, 201, 359–364. https://doi.org/10.1007/s00221-009-2028-6
Slater, M., Spanlang, B., Sanchez-Vives, M. V., & Blanke, O. (2010). First person experience of body transfer in virtual reality. PLOS ONE, 5(5), Article e10564. https://doi.org/10.1371/journal.pone.0010564
Slutsky, D. A., & Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. NeuroReport, 12(1), Article 7. https://doi.org/10.1097/00001756-200101220-00009
Tastevin, J. (1937) “En partant de l’experience d’Aristotle [Starting from Aristotle’s experience].” L’Encephale 1.57(84), 140–158.
Tsakiris, M. (2010) My body in the brain: a neurocognitive model of body-ownership. Neuropsychologia, 48(3), 703–712. https://doi.org/10.1016/j.neuropsychologia.2009.09.034
Tsakiris, M., Carpenter, L., James, D., & Fotopoulou, A. (2010). Hands only illusion: Multisensory integration elicits sense of ownership for body parts but not for non-corporeal objects. Experimental Brain Research, 204(3), 343–352. https://doi.org/10.1007/s00221-009-2039-3
Tsakiris, M., & Haggard, P. (2005). The rubber hand illusion revisited: Visuotactile integration and self-attribution. Journal of Experimental Psychology: Human Perception and Performance, 31(1), 80–91. https://doi.org/10.1037/0096-1523.31.1.80
Tsakiris, M., Haggard, P., Franck, N., Mainy, N., & Sirigu, A. (2005). A specific role for efferent information in self-recognition. Cognition, 96, 215–231. https://doi.org/10.1016/j.cognition.2004.08.002
van Dam, L. C. J., Parise, C. V., & Ernst, M. O. (2014). Modeling multisensory integration. In D. J. Bennett & C. S. Hill (Eds.), Sensory integration and the unity of consciousness (pp. 209–229). MIT Press. https://doi.org/10.7551/mitpress/9780262027786.003.0010
Wagenmakers, E. J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., & Morey, R. D. (2018). Bayesian inference for psychology. Part II: Example applications with JASP. Psychonomic Bulletin & Review, 25, 58–76. https://doi.org/10.3758/s13423-017-1323-7
Waltemate, T., Gall, D., Roth, D., Botsch, M., & Latoschik, M. E. (2018). The impact of avatar personalization and immersion on virtual body ownership, presence, and emotional response. IEEE Transactions on Visualization and Computer Graphics, 24(4), 1643–1652. https://doi.org/10.1109/TVCG.2018.2794629
Credit authors statement
Marion Giroux: Conceptualization, Formal analysis, Investigation, Methodology, Writing—first draft. Julien Barra: Conceptualization, Methodology, Writing—review and editing of the draft. Christian Graff: Writing—review and editing. Michel Guerraz: Conceptualization, Methodology, Supervision, Writing—first draft, Resources
Funding
The work was supported through funding from the University Savoie Mont Blanc (France).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
None of the authors have any conflicts of interests.
Additional information
Open Practices Statement
The raw data have been made available from the following address: https://doi.org/10.6084/m9.figshare.12906485.v3
Videos of the experimental conditions have been made available from the following address:
Experiment 1: https://doi.org/10.6084/m9.figshare.13359158
Experiment 2: https://doi.org/10.6084/m9.figshare.13359179
These studies have also been preregistered prior to conducting the research.
Experiment 1: https://osf.io/edjsm/?view_only=02550e36e8f44e0b89b627a06a9b4f50
Experiment 2: https://osf.io/39yfr/?view_only=8ed3a064ccaf40eabcd3df0bbaf9c863)
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Highlights
• Spatial congruence is considered to be determinant for multisensory integration.
• Kinaesthesia dependence on spatial congruence has been tested in virtual reality.
• Visual motion cues of an avatar seen in a mirror or from the 3rd-PP are integrated for kinaesthesia.
• Visual perspective affects the weight of the avatar's motion cues in kinaesthetic perception.
Appendices
Appendix 1: Embodiment questionnaire based on Gonzalez-Franco and Peck (2018)
All these questions were asked in both experiments, except for Question 4 and Question 5, which were only asked in Experiment 1.
“During the preceding experimental phase, there were moments in which . . . .”
Q1 – “I felt as if the virtual arms were my own arms”
Q2 – “It felt as if the virtual arms I saw were someone else’s arms”
Q3 – “It seemed as if I might have more than two arms”
Q4 – “I felt as if the virtual arms I saw when looking in the mirror were my own arms”
Q5 – “I felt as if the virtual arms I saw when looking at myself in the mirror were another person’s arms”
Q6 – “It felt like I could control the virtual arms as if they were my own arms”
Q7 – “The movements of the virtual arms were caused by my own movements”
Q8 – “I felt as if the movements of the virtual arms were influencing my own movements”
Q9 – “I felt as if the virtual arms were moving by themselves”
Q14 – “I felt as if my arms were located where I saw the virtual arms”
Q15 – “I felt out of my body”
Q17– “It felt as if my real arms were turning into ‘avatar’ arms”
Q18 – “At some point it felt as if my real arms were starting to take on the posture or shape of the virtual arms that I saw”
Q19 – “At some point it felt that the virtual arms resembled my own real arms, in terms of shape, skin tone, or other visual features”
Q20 – “I felt like I was wearing different clothes from when I came to the laboratory”
Appendix 2 Boxplots for each item of the embodiment questionnaire grouped by embodiment subcomponent (Ownership, Agency, Location, and External Appearance) for Experiment 1
Appendix 3: Boxplots for each item of the embodiment questionnaire grouped by embodiment subcomponent (Ownership, Agency, Location, and External Appearance) for Experiment 2.
Rights and permissions
About this article
Cite this article
Giroux, M., Barra, J., Graff, C. et al. Multisensory integration of visual cues from first- to third-person perspective avatars in the perception of self-motion. Atten Percept Psychophys 83, 2634–2655 (2021). https://doi.org/10.3758/s13414-021-02276-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13414-021-02276-3