Abstract
Purpose
Metaverse, that is extended reality (XR)-based technologies such as augmented reality (AR) and virtual reality (VR), are increasingly believed to facilitate fundamental human practice in the future. One of the vanguards of this development has been the consumption domain, where the multi-modal and multi-sensory technology-mediated immersion is expected to enrich consumers' experience. However, it remains unclear whether these expectations have been warranted in reality and whether, rather than enhancing the experience, metaverse technologies inhibit the functioning and experience, such as cognitive functioning and experience.
Design/methodology/approach
This study utilizes a 2 (VR: yes vs no) × 2 (AR: yes vs no) between-subjects laboratory experiment. A total of 159 student participants are randomly assigned to one condition — a brick-and-mortar store, a VR store, an AR store and an augmented virtuality (AV) store — to complete a typical shopping task. Four spatial attention indicators — visit shift, duration shift, visit variation and duration variation — are compared based on attention allocation data converted from head movements extracted from recorded videos during the experiments.
Findings
This study identifies three essential effects of XR technologies on consumers' spatial attention allocation: the inattention effect, acceleration effect and imbalance effect. Specifically, the inattention effect (the attentional visit shift from showcased products to the environmental periphery) appears when VR or AR technology is applied to virtualize the store and disappears when AR and VR are used together. The acceleration effect (the attentional duration shift from showcased products to the environmental periphery) exists in the VR store. Additionally, AR causes an imbalance effect (the attentional duration variation increases horizontally among the showcased products).
Originality/value
This study provides valuable empirical evidence of how VR and AR influence consumers' spatial bias in attention allocation, filling the research gap on cognitive function in the metaverse. This study also provides practical guidelines for retailers and XR designers and developers.
Keywords
Citation
Chen, J., Xi, N., Pohjonen, V. and Hamari, J. (2023), "Paying attention in metaverse: an experiment on spatial attention allocation in extended reality shopping", Information Technology & People, Vol. 36 No. 8, pp. 255-283. https://doi.org/10.1108/ITP-09-2021-0674
Publisher
:Emerald Publishing Limited
Copyright © 2023, Juan Chen, Nannan Xi, Vilma Pohjonen and Juho Hamari
License
Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
Humans are experiencing an unprecedented information technology revolution aiming to substitute and modify the perceived physical reality. The “metaverse” is no longer limited to science fiction and video games; it appears in almost all daily life settings, such as shopping (Bonetti et al., 2018), tourism (Kwok and Koh, 2020), training (Kaplan et al., 2020), education (Hughes et al., 2005) and interpersonal relationships (Wang et al., 2022). Currently, metaverse-related research and practices have mainly focused on virtual reality (VR), augmented reality (AR) and hybrid reality/mixed reality (MR), which are conceptually summarized under the umbrella term of extended reality (XR). The global XR market reached US$28 billion in 2021 and is expected to reach US$250 billion by 2028 (Statista, 2022). Despite the high expectations for the metaverse, such XR technologies and information systems are still mainly being adopted in the gaming industry. The majority of business practitioners are torn on applying VR and AR in retailing, consumption, marketing and advertising. Considering the financial and technical costs, they are unsure whether AR and VR can create similar or even better consumer experiences than in physical reality (Riar et al., 2022; Xi and Hamari, 2021).
One of the main concerns lies with the role of XR technologies in the cognitive aspects of consumer experiences, including the processes that consumers use to search, analyze, internalize and apply various information. Among these processes, being aware of information, namely, paying attention, is the prerequisite for information processing (Binetti et al., 2019; Dixon et al., 2014). In most shopping and business scenarios, individuals typically demonstrate attention by emphasizing the locations of perceptions with high salience; this process is called spatial attention (Cohen and Shoup, 2000). The recent empirical findings from Xi et al. (2022) indicate that consumers may present different cognitive patterns in extended realities (XRs), as the levels of subjective workload were reported differently between XRs and physical reality. Therefore, there is a fundamental research question of whether consumers use similar patterns in physical shopping environments to search and discover information, such as allocating attention in different XRs including VR, AR and reality combining augmented and virtual elements. It is also one of the important future research agendas suggested by Xi and Hamari (2021) regarding the effectiveness and efficiency of XR shopping.
Although the non-biased attention in XR shopping environments may be found in scattered studies (e.g. Pfeiffer et al., 2020), the increasing number of emerging evidence has indicated that the virtualization of the environment (e.g. VR) and the augmentation of information (e.g. AR) might greatly distract one’s spatial attention and lead to spatial bias in attention, such as toward the left or right hemisphere (Bartlett et al., 2020). But still, there is a need for granular analysis of attentional patterns by comparing different realities, technologies, interactions and areas of space, contributing to the development and extension of attention bias theory (see Section 2.2). Addressing such research questions is quite challenging since the findings are highly influenced by experimental design and the adopted measures. As metaverse retailing research is still in its infancy, most previous studies aimed to explore the benefits, advantages and new outcomes of employing XR in shopping via designing and providing a more interactive, functional and user-friendly system (Carlson et al., 2011; Ketelaar et al., 2018). Elucidating the differences in, e.g. the information amount, content and interactivity between conditions has not been the main focus. Such research paradigm also indicated in the two recently published literature review studies by Xi and Hamari (2021) and Riar et al. (2022), leads to concerns about affecting the internal validity of results related to attention, as external factors were not effectively controlled in quite a few experiment-based studies. It should also be noticed that most likely due to a lack of conceptualized understanding of the metaverse, the two core technologies, AR and VR, are rarely compared with each other in one single study, especially their differentiated effects on spatial attention and in-store attention biases. Such a comprehensive comparative investigation of physical reality and different extended realities is also important to deepen our understanding of the metaverse as a whole.
Therefore, the current study empirically examines the differences in consumers' spatial attention allocation in metaverse shopping by comparing the two main XR technologies — AR and VR. We conduct a 2 (VR: yes vs no) × 2 (AR: yes vs no) between-subjects laboratory experiment (N = 159). We build a brick-and-mortar store (as the control), and three XR stores are developed by employing a laser scanner to model the shop in a 1:1 scale, 3D environment and applying an AR image recognition program. Four spatial attention indicators — visit shift, duration shift, visit variation and duration variation — are compared based on attention allocation data converted from head movements observed in videos recorded during the experiment.
This study provides valuable empirical evidence on how VR and AR influence consumers' spatial bias in attention allocation. The findings fill the research gap in cognitive function in the metaverse and contribute to the interdisciplinary fields of information systems, human–computer interaction, consumer psychology and retail management. Moreover, this study provides practical guidelines for retailers embracing XR technologies, such as adapting their in-store display strategy to leverage consumers' attention. It also enlightens XR device designers and developers to help them improve technical solutions in business scenarios, especially retail.
The remainder of this paper proceeds as follows. Section 2 reviews research on XR technology and spatial attention in XR retail contexts and then develops hypotheses. Section 3 describes the methodology for applying spatial attention indicators based on video analysis. The results are presented in Section 4. Section 5 discusses the key findings and the theoretical and practical implications, while Section 6 concludes.
2. Research background
2.1 XR technologies
XR is an umbrella term encapsulating various computer technologies, such as VR and AR, that aim to substitute or modify the perceived present reality. VR has evolved from referring to a mere three-dimensional digital representation of the world (Lee and Chung, 2008) to the reality constructed in certain kinds of virtual games (Zyda, 2005). In VR environments, users can not only see detailed 3D scenarios but also manipulate avatars to explore the environment, navigate virtually and act on digital objects (Brooks, 1999). Nowadays, VR has been developed to almost fully immerse users in the virtual environment by exclusively isolating them from the physical world; more importantly, it replaces their “natural” perceptions of and interactions with physical reality with digitally-mediated perceptions and interactions (Berg and Vance, 2017; Bonetti et al., 2018). Specifically, VR technology creates a reality that digitally “substitutes” the perceived reality (Manis and Choi, 2019; Xi and Hamari, 2021; Yim et al., 2017), while AR technology superimposes all sensory information — for example, visual, sound, olfactory and haptics — over the perceived reality (Pantano et al., 2017; Rese et al., 2017; Xi et al., 2022). AR aims to “modify” or “augment” the perceived reality, no matter whether the reality is virtual or physical. Applying AR to augment and modify VR is called augmented virtuality (AV; Albert et al., 2014).
2.2 Spatial attention in XR shops
Attention, the selective allocation of cognitive resources to information exposure (Shiffrin and Schneider, 1977), is an essential aspect of individuals' psychology. Spatial attention allocation, or determining a locus of focus from specific location-based cues (Kim and Cave, 2001; Olshausen et al., 1993), is a critical component of attention allocation. Spatial attention allocation is the first-factor determining attention allocation to object-based features (Kim and Cave, 2001; Reuter-Lorenz et al., 1990; Shulman et al., 1985). Moreover, it correlates very closely with other cognitive processes, such as perception and memory; consequently, it influences people’s information-seeking, decision-making, learning and creating (Naert et al., 2018). In shopping, spatial attention allocation affects consumers' product evaluations (Schoormans and Robben, 1997) and influences their purchase intention (Chandon et al., 2009; Clement, 2007), thus making in-store spatial attention allocation a pertinent academic interest (Streicher et al., 2020).
Attentional bias is defined as an individual selectively allocating attention to certain stimuli while tending to ignore other stimuli (Ekhtiari and Paulus, 2016; Mitchell and Potenza, 2017). Researchers find factors of attentional bias, including multiple affective disorders such as anxiety that might lead to craving and addiction behavior (Shi et al., 2019), as well as environmental threats (McNally, 2019) and individuals' brain activities (Dickinson and Intraub, 2009). As XR technologies have extended individuals' existing place from physical space to metaverse, where their cognitive activities could be different from the physical space, human–computer interaction research has identified XRs could possibly generate attentional bias, which basically means distortions of the attention compared to physical reality (Horvitz et al., 2003; McCrickard and Chewar, 2003). The research covers various scenarios, such as XR driving (e.g. Medenica et al., 2011), XR learning (e.g. Huang et al., 2019) and XR exercising (e.g. Mestre et al., 2011, Table 1). The identified bias effects include the disassociation effect of VR (Maringelli et al., 2001), the tunnel effect of AR as exogenous-stimuli concentration (Dixon et al., 2014) and the peripersonal-space concentration (Binetti et al., 2019). This kind of attentional bias is called XR-led bias in this current study.
However, studies in the XR retail context are scarce compared with research in other XR contexts (Table 1). Research in XR retail still includes disputes, ambiguity and a lack of synthesis of XR’s effects. Most research suggests that a similar attention pattern (i.e. no bias exists) is utilized in XR stores and in brick-and-mortar stores (e.g. Pfeiffer et al., 2020). Only a few studies have echoed biases that are found in other XR scenarios (i.e. a concentration effect in AR stores; Yang et al., 2020). The current study argues that XR’s effects on consumers' spatial attention allocation require further validation. First, research has been conducted under CAVE VR conditions (e.g. Pfeiffer et al., 2020) and under hand-held mobile AR conditions (e.g. Wychgel, 2020; Yang et al., 2020). These technical conditions limit participants' natural navigation and interaction in the XR environment. Whether XR biases consumers' spatial attention in stores that allow consumers' natural navigation and interaction, is still unknown. Furthermore, granular analyses of attention are lacking. Most studies focus on XR’s effects on consumers' attention allocation for specific areas of interest (AOIs; e.g. Pfeiffer et al., 2020; Wychgel, 2020; Yang et al., 2020), and the AOI is usually limited to a small range of the retail space (e.g. a shelf or signage). Whether XR biases consumers' spatial attention in XR stores' overall spatial hierarchy is still uninvestigated.
Moreover, most studies aimed to investigate a sole XR technology component (VR or AR). However, considering that VR “replaces” the original physical reality with its virtualization, while AR “superimposes” a virtual layer on the original reality, VR and AR might have different impacts on consumers' spatial attention allocations in XR stores. The combination of VR and AR could have effects on attention that are different than those created through their individual use. Thus extant research leaves room for a granular analysis based on comparing/integrating attention bias led by VR and AR technology.
Based on the above considerations, this study explores the reasons for the diversity of XR technologies' biasing effects on consumers' spatial attention allocation in retail locations. Specifically, the experimental conditions strictly control the stimuli to which participants are exposed and allow them to navigate in and interact with the environment. The researchers recorded participants' in-store behavior and analyzed their attentional visits and duration toward different AOIs in different XR stores. Thus, this study offers an overview and valuable explanations of consumers' XR-led attention biases.
2.3 Hypotheses development
The cognition resource allocation and attention competition theories (i.e. the biased competition theory, see Desimone and Duncan, 1995; Yantis, 2000) provide a basic framework for understanding how individuals allocate their attention both on space-based cues and object-based cues. Two paths describe how space-based cues and object-based cues compete for individuals' cognition resources: the bottom-up path describes the cues with more prominence (e.g. the colors, outlines or exogenous cues) wins more attention; the top-down path depicts the cues could receive more attention than their competitors due to the individual’s goals, aims and motivations (Briand, 1998; Goldsmith and Yeari, 2003; Yantis, 2000). Specifically, spatial attention has been described as a flashlight scanning the space under the individual’s goals. During this process, some prominent cues might distract the scan and draw attention elsewhere (Treue, 2003; Wolfe, 1998). Besides, according to cognitive load theory, distraction, as well as information complexity, might add to extraneous cognitive load to that individual (Sweller, 2011), both of which negatively impact the information processing efficiency, whereas germane cognitive load positively influence the information processing efficiency (Sweller, 2011). Additionally, extraneous cognitive load also influences the neuro functions of the left and right hemispheres, which further affects the spatial allocation of attention (Massara et al., 2014).
This current study contextualizes this general picture of spatial attention allocation into the in-store shopping scenario and highlights XR technologies' effects on spatial attention allocation. First, consumers' in-store spatial attention includes two goals: they observe and absorb the shop’s ambiance, and they examine products displayed in the shop (Otterbring et al., 2014, 2016; Riva et al., 2009). Therefore, consumers allocate their cognitive resources among different AOIs (i.e. the showcased product area and the environmental periphery); their attention remains on those AOIs and shifts between them. Such attention dynamics create different magnitudes of in-store spatial attention: attentional visit and attentional duration (both of which shift between the showcased product areas and the environmental periphery and vary across showcased product areas). Then, as shown in Figure 1, VR and AR impose possible influences on both the top-down and bottom-up processes (specifically, their goal shifting, the task complexity and the distraction) of consumers' cognition resource allocation and cognitive load. Thus consumers' spatial attention allocation might be biased in both magnitudes compared to the non-VR or non-AR circumstances. Section 2.3.1–2.3.4 deduct in detail those XR-led biases.
2.3.1 XR’s effects on the attentional visit
In VR, users' sensory perceptions of physical reality are overwhelmingly substituted by sensory perceptions of its digital replica. Such a condition arouses users' caution when they are in motion (Chen et al., 2015). According to the biased competition theory, each stimulus in the space competes for the individual’s cortical representation and the cognitive process, including attention (Desimone and Duncan, 1995). Further, as the integrated model of attention elucidates, attention can be “captured” by the physical features of stimuli and “directed” toward an item according to the goals of individuals (Yantis, 2000); that is, individuals' goals, motivations and current tasks decide which stimuli they pay attention to and which they neglect (Briand, 1998; Goldsmith and Yeari, 2003). Thus, consumers conduct more dynamic attentional visits toward peripheral environmental elements that work as guide-for-navigation references (Otterbring et al., 2014, 2016) in VR than in the non-VR environment. Therefore, VR increases the attentional frequency of the environmental periphery while having no impact on the attentional frequency of the products. Thus, we posit the following hypothesis:
Compared to the non-VR store, the frequency of consumers' allocated attention to the showcased products is lower in the VR store.
By augmenting a virtual layer of extra information onto the product, AR presents exogenous stimuli in consumers' peripersonal space. Exogenous stimuli differ from the currently attended stimuli (the endogenous stimuli), as being the interruption of individuals' attention (Yantis, 1993). According to the integrated model of attention, exogenous stimuli, are more likely to capture consumers' attention than endogenous information (Binetti et al., 2019; Dixon et al., 2014). Thus, AR retains consumers' attention on the focal product area, which increases the attentional duration while decreasing the attentional frequency of products. Therefore, we have the following hypothesis:
Compared to the non-AR store, the frequency of consumers' allocated attention to the showcased products is lower in the AR store.
Regarding attentional visit variation, namely the frequency of the spatial attention allocated across different areas of the showcased products, consumers in VR stores would likely present similar allocations in the brick-and-mortar store (Pfeiffer et al., 2020). Since AR functions as a digitalized presentation of product information and provides it spatial-correlated, there is no explicit evidence that AR would significantly influence such consumers' attentional visit variations among different product areas. As Wychgel (2020) points out, there is no significant difference in the attentional visit to different AOIs between AR and traditional information presentation in the store. The (null) hypotheses of the study state there to be no effect on attentional visit variation stemming from VR or AR when the environment and information have been 1:1 replicated as close to match a “real” scenario (as operationalized as the control group of the experiment). Therefore, we propose the following (null) hypotheses.
There is no significant difference between VR and non-VR stores on the frequency regarding consumers' allocated attention across different areas where showcased products are.
There is no significant difference between AR and non-AR stores on the frequency regarding consumers' allocated attention across different areas where showcased products are.
2.3.2 XR’s effects on the attentional duration
According to cognitive load theory, information processing efficiency is highly associated with individuals' cognitive load. However, the three types of cognitive load affect information processing efficiency differently: germane load positively influences information processing efficiency, while increases in extraneous and intravenous load make people’s information processing less efficient (Sweller, 2011). The literature shows that both VR and AR influence users' cognitive load in different dimensions (i.e. germane and extraneous loads). VR increases consumers' germane load as the digitization of everyday scenarios makes consumers more aroused and interested (Sweller, 2005). Individuals' information processing efficiency further influences their attentional duration. For instance, if the individual is less efficient when processing the information within their locus, they will spend more time attuned to that locus (Rayner, 1998). The following hypothesis is posited:
Compared with the non-VR store, the time spent regarding consumers' allocated attention to showcased products is lower in the VR store.
As for AR, providing additional spatially correlated information for the showcased products will improve episodic memory (Hou et al., 2013), increase the germane load and lift information processing efficiency. However, AR also increases the extraneous load, otherwise decreasing the information processing efficiency. Since AR superimposes a digital layer onto an object in physical reality, it increases the information bandwidth, which requires users to merge different channels of information (the information about that object and its digital augmentation) mentally (Cuomo et al., 2020; Sweller, 2005), and consequently increases consumers' extraneous cognitive load. Additionally, while in the AR stores, users are usually required to use hand-held devices or headsets to acquire augmenting information, the difficulty of operating AR systems and relatively longer response time for providing digital information might lead to more time spent on the showcased products. Altogether, we posit the following hypothesis:
Compared with the non-AR store, the time spent regarding consumers' allocated attention to showcased products is higher in the AR store.
Previous research (Berton et al., 2019; Pfeiffer et al., 2020) acclaims that the attentional duration is similar in VR to that in physical reality. Specifically, the HMD-VR setup is more in line with the physical reality situation than the other VR setups, such as desktop VR or CAVE VR (Berton et al., 2019). Thus, we propose the following (null) hypothesis.
There is no significant difference between VR and non-VR stores on the time spent regarding consumers' allocated attention across different areas where showcased products are.
However, we find some interesting discussions associated with extraneous cognitive load and the left–right symmetry of information processing, which could help hypothesize AR’s effect on attentional duration variation. As Massara et al. (2014) indicate, the more complicated task (generating a higher extraneous cognitive load) yields a more intense left–right symmetry in attention allocation. Due to the division of their different functions, the right and left hemispheres are influenced at different levels when the extraneous cognitive load increases (Loftus and Nicholls, 2012). Thus, individuals process stimuli on the left side and the right side at different efficiency levels, and an asymmetry in rightward and leftward attention emerges (Bartlett et al., 2020; Tucker et al., 1999). Presumably, with the increase of extraneous cognitive load, AR might intensify the horizontal, left–right attention asymmetry, thus increasing attentional duration variation. On considering this, the following hypothesis is posited:
Compared with the non-AR store, the time spent is more unevenly in the AR store regarding consumers' allocated attention across different areas where showcased products are.
3. Methodology
3.1 Research design
To investigate the effects of XR technology (i.e. VR and AR) on attention (i.e. shift and duration) in the multi-dimensional store space, we deployed a 2 (VR: yes vs no) × 2 (AR: yes vs no) between-subjects experiment design and also built four versions of the second-hand LP record store (a non-XR bricks-and-mortar store, a VR store, an AR store and an AV store). 159 participants were randomly assigned to one of the four stores where they had 10 min to spend 10 euros optimally based on their preferences.
Measuring participants' attentional visits can offer a more accurate view of their attention allocation than indirectly measuring their performance on attention-related tests or their self-reported attention status post hoc (Meißner et al., 2019). The most common methods of collecting participants' attention allocation data include eye-tracking and video analysis methods (Dixon et al., 2014; Mestre et al., 2011). The eye-tracking method directly captures participants' eyeball movements in response to visual stimuli (i.e. gazes). Although it collects more accurate data on participants' attention than video analysis, there are a few limitations of using current eye-tracking technologies for XR research which is widely applied to detect users' attention in the 2D field: The accuracy would be vastly decreased when participants are freely moving and navigating (Meißner et al., 2019; Rappa et al., 2019); the usability of XR system might be influenced which would further influence the overall user experience (Meißner et al., 2019); it is difficult to track depth component of the 3D stereo attention point (Rappa et al., 2019). In this study, we investigate how spatial attention is allocated to four main areas of different in-store environments rather than specific shelves or products. As an alternative method, video analysis denotes participants' attention allocation behavior by their head movement directions, which is sufficient and feasible for this study. Therefore this present study chooses video analysis. We use cameras to record each participant’s in-store behaviors, including their head movements which become the basis of the attentional visit and duration annotation.
3.2 Research setting
3.2.1 Setting of the non-XR shop
We built a “bricks-and-mortar music store” (4.24 m × 5.09 m = 21.6 m2) at the campus of one university in northern Europe. We set shelves symmetrically on the three walls: walls on the two sides and opposite the entrance (abbreviated as left, right and middle walls). Each shelf had three layers. The products sold in the stores were second-hand English LP records, which were placed cover-outside on those layers. There were 54 different records in the shop altogether (18 records on each shelf), accompanied by a product information sheet (providing information about the artists, releasing company, playlist, social network rating and comments). In this study, we used second-hand LP records as products containing specific visual and text information. Consumers need to search for that information when making purchase decisions (similar to Perron-Brault et al., 2020). Compared to other everyday product categories, most participants were unfamiliar with second-hand LP records; thus, we could exclude prior product preferences' disturbance. Besides, second-hand LP records allow a large number of similar products with different information to be “sold” in the store and be “bought” by participants (they would take the LP records as compensation for participating in the experiment).
3.2.2 Building of XR stores
The VR store environment was realized by the Unity software as a 3D-constructed replica of the bricks-and-mortar store (as shown in Appendix Figure A1). Participants wore the Valve index headset and used its controllers during the experience. The headset provides high-fidelity VR experiences among the current consumer-grade VR head-mounted displays. The Valve Index Controllers allow participants to have lively interaction experiences, including intuitively reaching out, grabbing, throwing or dropping the product directly, tracking their hand and finger position, motion and pressure. Besides, the printed product information sheets were digitally duplicated in the VR-mediated condition.
In the AR version of the shopping experience, instead of the physical-paper product sheets, product information was displayed to consumers through the lens on the head-mounted display device (Microsoft Hololens 1). When participants looked at one record, the related information page would pop up on the lens.
The AV store (the experimental condition combining VR and AR in this study) presents the product information as augmented reality objects overlaid on the 3D-constructed replica of the bricks-and-mortar store (similar to the VR store). Participants also wore the Valve index headset and used its controllers during the experience. The head position tracking script in the Unity software allowed participants to see the popped-up product information sheet while orientating a specific product.
3.2.3 Video recording apparatus
Researchers placed two web cameras (Mi home security 360 cameras) on the wall’s two ceiling corners in the store’s physical space (participants in VR and AV conditions navigate the same physical place of the bricks-and-mortar/AR store). The aim is to ensure that the video can fully record participants' in-store behavior in stereoscopic vision. Camera 1 was placed on the left and Camera 2 on the right, facing the store entrance. The cameras recorded participants' body movements and interactions with the products during the ten-minute shopping time. Screens of VR and AV programs facilitating participants' shopping were also recorded as supplementary video materials. Thus, this study collected a total of 67 h of video recordings.
3.3 Participants
Between September and November 2019, we recruited student participants who had a normal or corrected-to-normal vision from the same university where the stores were built. In total, 265 students applied to participate; out of which 165 finally were booked and participated in the experiment. We collected participants' personal information (gender, age, educational background, income and VR experience) during the recruitment. Among them, six participants were dropped for the following reasons: two participants were disqualified for not being university students; one participant experienced technical difficulties with the VR headset and stopped the experiment; another three participants' videos were recorded incompletely due to technical failures. Therefore leaves a remaining sample of N = 159 (45.9% female, 55.9% 20–24 years old, 56.6% undergraduate students, 57.8% studying engineering and technology, 55.9% less than 499 euros monthly income, diverse in nationalities such as Finland, China, Germany, Vietnam, Spain, Russia, India, France, etc.). The experiment design and implementation adhered to the Finnish National Board on Research Integrity TENK Guidelines 2019.
3.4 Experiment procedure
We randomly assigned each participant to one of the four stores. Before participating in the study, they all signed the consent form. All participants were then given instructions for the experiment and a tutorial about wearing and using the headsets and controllers. Their visions in XR devices were adjusted to a satisfactory level before the shopping task [1]. Each participant was asked to complete a ten-minute shopping by navigating, browsing, selecting and making a purchase decision for music products in the shop with the given 10 euros gift card.
3.5 Video analysis
We recorded participants' 10-min in-store behavior. Appendix Figure A2 shows an example of the video snapshots belonging to one participant. We annotated the videos manually according to participants' head direction dynamics towards different locations. We used BORIS (Behavioral Observation Research Interactive Software) to subtract quantitative timed-event data (Friard and Gamba, 2016), that is attentional visit and attentional duration. The data were then calculated into four attention indicators: attentional visit shift, attentional duration shift, attentional visit variation and attentional duration variation.
Defining the coding scheme. This study mainly measured participants' attention allocation by analyzing their head directions towards a hierarchy of locations: AOIs (being further distinguished as shelves located at the left, middle and right walls, separately) and non-AOI (i.e. environmental periphery). Therefore, naturally, the coding scheme (a mutually exclusive and exhaustive set of coding, Bakeman and Quera, 2011) of participants' in-store attention allocation behavior was established as: “AOI-Left,” “AOI-Middle,” “AOI-Right,” and “non-AOI” (refer to Appendix Table A1). Then annotation was conducted based on this coding scheme.
Timed-event annotation. Annotation began at the very beginning of the video. When the participant shifted their attention toward another location, we triggered the corresponding command key on the keyboard. We merged any event shorter than one second with the following event [2].
Collecting quantitative data. Then we transformed the BORIS records into the event-based (i.e. attentional visit) and time-based (attentional duration) quantitative data towards different in-store locations. The attentional visit, defined as the total times the participant looked at each specific in-store location, was collected when the researcher triggered the key command. The attentional duration towards each exact in-store location, defined as the total time duration the participant spent on looking at the specific location, was also collected by the time budget tool of BORIS by adding up the entire time duration of each attentional visit towards that location.
Calculating attention indicators. Next, four attention indicators (attentional visit shift, attentional duration shift, attentional visit variation and attentional duration variation) were calculated according to the formulas provided in Section 3.6.
3.6 Measurements of spatial attention allocation
To provide more accurate measurements and granular analysis of spatial attention allocation, we divide the in-store space into AOIs (i.e. the shelves with the showcased products) and non-AOI (i.e. the environmental periphery). We operationalized spatial attention allocation as indicators based on head pointing direction toward different locations (shown in Appendix Table A2). The indicators represent different dimensions of spatial attention allocation. Spatial attention allocation can either be captured by attentional visit numbers (i.e. event-based indicators) or by the time duration (i.e. time-based indicators). We can either examine the attentional shift from AOIs to non-AOI (and vice versa) or further examine the location variations of attention among different AOIs.
Indicators of attentional visit shift and attentional duration shift were calculated according to Formula (1) and (2), separately:
Indicators of attentional visit shift variation and attentional duration variation were calculated according to Formula (3) and (4), separately:
In which i denotes each AOI, i = 1, 2, …, m; m is the total number of AOIs (m = 3 in this case); j denotes the non-AOI.
4. Analyses and results
4.1 Descriptive results
We include participants' gender, age, educational background, income and VR experience as control variables. Descriptive results for control variables are shown in Appendix Table A3. We conduct the one-way analyses of variance (Appendix Table A4) and the multiple comparison analyses on control variables, which report no significant differences across different groups, indicating the experiment achieves balance and randomization. Descriptive results of the four attention indicators (refer to Table 2) show differences exist between the non-XR store and different XR stores. Specifically, as for the attentional visit shift, the none-XR group (M = 0.800, SD = 0.073) is the highest, followed by the AR group (M = 0.755, SD = 0.078), AV group (M = 0.754, SD = 0.077) and VR group (M = 0.734, SD = 0.075). Besides, attentional visit variation in the group of AV is the largest (M = 0.378, SD = 0.175), followed by groups of VR (M = 0.348, SD = 0.184) and AR (M = 0.322, SD = 0.194) and non-XR (M = 0.308, SD = 0.159) group is the smallest. As for the attentional duration shift, it was the highest in the non-XR group (M = 0.946, SD = 0.042), followed by the AR group (M = 0.943, SD = 0.057), VR group (M = 0.924, SD = 0.054) and AV group (M = 0.908, SD = 0.066). Attentional duration variation in the AR group is the largest (M = 0.401, SD = 0.230), followed by groups of AV (M = 0.396, SD = 0.214), VR (M = 0.335, SD = 0.205) and the non-XR group (M = 0.305, SD = 0.142) is the most limited.
4.2 Analyses of variance
We conducted a series of two-way factorial analyses of variance (ANOVA) to investigate whether AR and VR influence consumers' spatial attention allocation (by four attention indicators). AR and VR consisted of two levels (yes = 1 and no = 0). Control variables were introduced as covariables in ANOVA. The significance level of 0.05 was used for all statistical tests.
4.2.1 Prior tests
Prior to conducting the ANOVA, we tested the equality of the error variance within each group and the normality of the residual among the groups. Under the dependent variable of attentional visit shift, Levene’s test showed equality of error variances (p = 0.978); Shapiro–Wilk’s test showed normality of the residual of the group of non-XR (p = 0.444), VR (p = 0.333) and AV (p = 0.475). Although the normality of the AR group was declined (p = 0.018), ANOVA could still be deployed due to its insensibility if normality is not satisfied (Lix et al., 1996). Similarly, conditions under the dependent variable of the other three attention indicators are considered feasible for deploying ANOVA. Table 3 summarizes ANOVA results of XR’s effects on the four attention indicators, separately. As for control variables, except for income having a significant positive effect on attentional visit shift (F (1, 155) = 11.201, p = 0.001), none of the control variables significantly affected any of the attention indicators.
4.2.2 XR's main effect on the attentional visit
The main effect of VR on attentional visit shift was significant, F (1, 155) = 8.248, p = 0.005, such as the attentional visit shift was significantly lower with VR (M = 0.744, SD = 0.076) than that without VR (M = 0.777, SD = 0.079). VR explained 5.2% of the variance of attentional visit shift. The result indicated that in stores with VR, the frequency of attention allocated to the product areas was significantly lower. Therefore H1VR was supported. On the contrary, the main effect of AR on attentional visit shift was non-significant, F (1,155) = 0.905, p = 0.343, such as the attentional visit shift with AR (M = 0.754, SD = 0.077) was not significantly different from that without AR (M = 0.768, SD = 0.081). The result implied that the frequency of attention allocated to the product areas in stores with AR is similar to that in non-AR stores. Thus H1AR was unsupported. Additionally, the interaction between AR and VR on attentional visit shift (F (1, 155) = 9.558, p = 0.002) is significant, which was further analyzed in Section 4.3.
The main effect of VR on the attentional visit variation was non-significant, F (1, 155) = 2.769, p = 0.099, such as the attentional visit variation with VR (M = 0.363, SD = 0.179) was not significantly different from that without VR (M = 0.315, SD = 0.176). The main effect of AR on the attentional visit variation was non-significant, neither, F (1,155) = 0.379, p = 0.539, such as the attentional visit variation with AR (M = 0.349, SD = 0.186) was not significantly different from that without AR (M = 0.327, SD = 0.171). Those results indicated there was neither significant difference in the frequency of consumers' attention allocation on the product areas in between VR and non-VR stores nor in between AR and non-AR stores. Accordingly, H2(0)VR and H2(0)AR were both supported. Additionally, neither was the interaction effect between VR and AR on the attentional visit variation, F (1, 155) = 0.009, p = 0.923.
4.2.3 XR's main effect on the attentional duration
The main effect of VR on the attentional duration shift was significant, F (1, 155) = 11.824, p = 0.001, such as the attentional duration shift was significantly lower with VR (M = 0.916, SD = 0.060) than without VR (M = 0.945, SD = 0.050). VR explained 7.4% of the variance of attentional duration shift. The result indicated that in stores with VR, the time consumers spent regarding attentional allocation on the product areas was significantly less. Thus H3VR was supported. On the contrary, the main effect of AR on the attentional duration shift was non-significant, F (1, 155) = 1.576, p = 0.211, such as the attentional duration shift with AR (M = 0.926, SD = 0.063) was not significantly different from that without AR (M = 0.935, SD = 0.049). Therefore, the time consumers spent regarding attention allocation on the product areas in stores with AR is similar to that in stores without AR. Thus H3AR was unsupported. Additionally, the interaction effect between AR and VR was non-significant, F (1, 155) = 0.204, p = 0.653.
The main effect of VR on the attentional duration variation was non-significant, F (1,155) = 0.071, p = 0.790, such that the attentional duration variation with VR technology (M = 0.366, SD = 0.210) is not significantly different from that without VR (M = 0.353, SD = 0.196). The result implied that, regarding the spending time, the attention consumers allocate across different product areas in stores with VR is similar to that in the non-VR stores. Thus H4(0)VR was supported. However, the main effect of AR on the attentional duration variation was significant, F (1,155) = 3.961, p = 0.048), such that the attentional duration variation with AR technology (M = 0.399, SD = 0.221) is significantly higher than that without AR (M = 0.320, SD = 0.174). AR explained 3.8% of the variance of the attentional duration variance indicator. The result indicated that, regarding spending time, consumers would allocate attention across different product areas more unevenly in stores with AR. Thus H4AR was supported. Additionally, the interaction between AR and VR was non-significant, F (1, 155) = 0.435, p = 0.510.
4.3 Pairwise comparisons
According to the ANOVA results, there is a significant interaction effect between AR and VR on attentional visit shift; thus, to further illustrate this interacting effect, the simple main effects analysis (SIDAK) was conducted using the EMMEANS syntax command within SPSS 18.0. As Figure 2 and Table 4 show, in the non-VR condition, there exists a significant difference between non-AR and AR (mean difference = 0.045, p = 0.008); however, in the VR condition, there was no significant difference between non-AR and AR (mean difference = −0.020, p = 0.249). Thus we get complementary information for the non-significant main effect of AR on attentional visit shift: AR significantly decreases consumers' frequency of attention on the product areas only in the environment not applied to VR technology; in a VR-mediated environment, AR causes no significant influence on consumers' frequency of attention on the product areas.
Beyond that, in the non-AR condition, VR significantly decreases the level of attentional visit shift than without VR (mean difference = 0.065, p < 0.001); however, in the AR condition, there was no significant difference between non-VR and VR (mean difference = 0.001, p = 0.963). We can infer that VR technology’s effect in decreasing consumers' frequency of attention on the product areas is no longer significant when being applied to the AR-mediated environment.
5. Discussions and implications
5.1 Key findings
This current study provides empirical materials for exploring how spatial attention is allocated in different XR environments and how XR technologies bring spatial attention biases. Table 5 further illustrates those explorations. XR-led biases are manifested as three kinds of effects — the inattention effect, the acceleration effect and the imbalance effect. Discussions of how those findings correlate and differ from previous research are presented in the following.
5.1.1 The inattention effect
First, the results show that VR decreases consumers' attentional visit shift. This reflects that VR generates a weighted decrease in consumers' attention being allocated to AOIs and a weighted increase in their attention to non-AOIs. We call this the inattention effect in reference to the shift in consumers' in-store attentional visit from the showcased products to the environmental periphery, which occurs because VR causes consumers to divert their attention from shopping to navigation.
Notably, previous research finds no significant difference in individuals' attention allocation in VR compared to physical reality (e.g. Berton et al., 2019; Hillaire et al., 2009; Pfeiffer et al., 2020) and mentions no similar inattention effect. These studies and our work differ in affordability for participants' navigation and interaction. Therefore, we suggest that when compared to navigation- and interaction-constrained VR scenarios (in which participants do not need to allocate their spatial attention to navigation needs), the navigation- and interaction-enabled VR scenario might create the inattention effect.
For AR, despite its significant main effect on attentional visit shift, the pairwise comparison uncovers the details of the effect. AR has an inattention effect when it is applied to physical reality environments (i.e. the AR store). However, when using AR in the VR environment (i.e. the AV store), consumers' attentional visit shift is no longer significantly influenced by AR. That is, the inattention effect of AR has a boundary: when VR coexists with AR, that effect diminishes. Such a situation further indicates that combining VR and AR could possibly create an attention allocation that is more like physical reality than are existing alternatives.
5.1.2 The acceleration effect
The results also show that VR decreases consumers' attentional duration shift, which leads consumers to spend relatively little time focusing on AOIs. This relative decrease in attentional duration toward AOIs implies the increase in germane cognitive load (and arousal and interest) leads to the increase in the speed of processing information on the showcased products; thus, we call it the acceleration effect. Previous studies have also indicated such an effect in interactive learning and cognitive therapies (Grealy et al., 1999). However, AR does not have an acceleration effect. The possible explanation might be that, for the AR-mediated environment, the positive effect of germane load on information processing efficiency compensates the negative effect of extraneous load and device operation difficulties.
5.1.3 The imbalance effect
The current study also finds that AR increases the attentional duration variation; that is, AR disturbs the balance in consumers' efficiency in processing information and leads to an attention imbalance. Accordingly, we call this the imbalance effect. Such an effect echoes previous studies in contexts other than retail (e.g. a left–right attentional asymmetry found in AR maze navigation; Bartlett et al., 2020). Beyond those studies that solely involve navigation scenarios, the present study generalizes AR’s imbalance effect by extending it to the shopping scenario. VR does not add substantially to consumers' spatial attention allocation imbalance. Altogether, the results indicate that, when it comes to spatial attention (im)balance, AR should be applied cautiously.
5.2 Research implication
First, this study makes a great attempt to develop the theoretical understanding of cognitive aspects regarding information seeking and awareness in the metaverse-oriented business field by focusing on consumers' attention in a metaverse shopping context. The majority of the existing studies on metaverse business are mainly based on the available applications in the market (e.g. AR apps, 360° video content and 3D product presentations). The understanding towards user experience and behavior is still insufficient since research questions were often limited to what features such XR applications contain. Therefore, many foundational research questions, such as new and unexpected patterns in information processing and cognitive function in the metaverse, are still not theoretically and empirically addressed; especially in one research mainstream, XR is considered as a substitute platform for investigating the traditional phenomena. By self-developing 1:1:1 digitally duplicated XR shopping environments based on the self-building physical store, the current study based on rigorous experiment method provides important and strong empirical evidence for the existence of different patterns in cognitive process in XR-led business. Thus, this study can enlighten the possible direction on investigating new dimensions of consumer experiences.
Second, the current study also enriches the existing XR shopping literature by contextualizing consumer-grade XR technologies to daily shopping activities. The two recently published systematic literature review studies by Riar et al. (2022) and Xi and Hamari (2021) have shown that despite the richness of discussions on the technological solutions for XR stores, investigations of consumer experiences and responses to XR retail options are insufficient and still at an exploratory stage (Alcañiz Raya et al., 2019; Hollebeek et al., 2020). Thus, the current study offers vivid knowledge of how consumers would cognitively respond to XR retail solutions by changing their spatial attention allocation during daily shopping activities such as navigation and searching for products. It also depicts how consumers experience attention acceleration and bias when XR retail solutions arouse cognitive load changes.
In addition, this current study fills out the gap in the current literature in which different XR technologies, such as VR and AR, have mainly been investigated separately. Due to the lack of comparison between different XR technologies, there is limited knowledge of the different effects of various XRs on consumer experiences such as cognition, emotion and behavior. Therefore, there are still barriers to the consensus on definitions and conceptualized understanding of VR, AR, MR and XR. This study first synthesized the definitions and findings regarding two core XR technologies – VR and AR and clarified the conceptualized differences between the two based on the literature review. By developing and examining hypotheses, this study further provided experimental evidence that AR and VR did have different effects on consumers' cognitive experiences, especially spatial attention allocation (attentional shift and duration). The current study also provides theoretical and methodological guidance for future research on how to conduct comparative studies.
Last but not least research contribution in this study is that based on cognition resource competition and cognitive load theory, four important indicators measuring the summation and variations of spatial attentional allocation were developed and examined. Attentional visit shift and attentional duration shift were measured to understand how much attention consumers would allocate to AOIs (vs non-AOI) during in-store shopping, which is the key objective in retailing management. More importantly, measuring the difference in attention allocation among AOIs (left vs middle vs right) provides more detailed and accurate information on revealing various cognitive patterns in different shopping environments. Therefore, the developed measures contribute to the retailing management literature as well as provide a valuable measurement tool for future research.
5.3 Practical implication
This study provides an enriched understanding of consumers' attention allocation for retailers and brands embracing XR technologies. First, retailers should gain full awareness of the trade-offs between cognitive and psychological outcomes of different XR solutions when virtualizing their bricks-and-mortar locations. Retailers should especially highlight the possible biases (e.g. the inattention effect, acceleration effect and imbalance effect, as illustrated in this study) in consumers' spatial attention allocation. Further, retailers should adapt their in-store display strategy accordingly to different XR retail solutions. Display strategies developed in the bricks-and-mortar stores might be inefficient in XR stores. New rules of consumers' attention allocation are emerging, and retailers should be aware of recent display cost–outcome schemes. For instance, display fees across horizontal locations should be reevaluated due to attention allocation imbalances. They should readjust in-store display arrangements, for example, to utilize peripheral spaces in addition to showcased spaces for new commercial opportunities such as in-store advertising.
Moreover, this study offers helpful directions for XR program developers and device designers to improve XR retail solutions. For instance, XR program developers should thoroughly consider consumers' in-store navigation, product interaction and different types of cognitive load when designing the human–computer interactions of XR stores. For instance, the intensity of consumers' in-store navigation and product interaction could influence the severity of attentional bias, especially shifting consumers' attention from the showcased products to the environmental periphery. With this in mind, XR program developers could consider integrating VR and AR to facilitate very realistic attention allocation. Additionally, XR device designers should fully consider the devices' virtual image presentation, navigation assistant approaches and cognition load when using them.
6. Conclusion, limitations and future directions
This study examines the effects of different XR technologies on consumers' spatial attention allocation. VR and AR, being applied individually, cause attentional visits to shift from showcased products to the environmental periphery (i.e. the inattention effect) while combining VR and AR diminishes the inattention effect. VR alone generates more attentional duration to be allocated from showcased products to the environmental periphery (i.e. the acceleration effect), while AR significantly increases the attentional duration variation horizontally among the showcased products (i.e. the imbalance effect).
However, this study is limited in several aspects that could provide new directions for future research. First, by focusing on gaining a granular view of consumers' spatial attention allocation in XR stores, the scope of the current study is constrained to comparing spatial attention allocation across extended realities. Considering that spatial attention allocation is vital in the shopping context (i.e. to specific shopping outcomes, such as product evaluation or purchase decisions), future studies could examine whether the relationships between spatial attention allocation and those outcomes are the same in XR stores as in the bricks-and-mortar store. Moreover, this study measured spatial attention allocation by analyzing the visit and duration of gaze towards different areas divided horizontally (left, middle, right and other areas). Future research could investigate other important indicators, such as attention allocation in vertical directions or different depth planes. Aside from spatial allocation, other facets of attention allocation, such as temporary allocation (Lutz et al., 2008) and the allocation on endogenous/exogenous stimuli (Goldsmith and Yeari, 2003) or different information formats (e.g. image/text; Lee et al., 2018) could also be included in the research scope to enrich the study of XR retail. The present study places more emphasis on visual attention; however, future research should explore attention allocation involving multiple sensory modalities (Massiceti et al., 2018), such as whether consumers' attention allocation is influenced by the (in)congruence of visual and auditory stimuli provided by XR.
Second, this study uses a university-student sample because students perform better as experimental controls (e.g. the experiment outcomes are less likely to be influenced by participants' physical health, cognitive capability and other unidentified factors). However, future studies could use broader samples to improve external validity (Sears, 1986). Since spatial attention allocation might differ across age groups (Hartley, 1993) or genders (Huang, 2018), a comparison of samples of these different groups should be considered.
The third limitation stems from the attention analysis method used in this study. This current study tracks participants' attention through their head direction by examining video recordings of participants' in-store behavior rather than through their eye-ball movement (i.e. by eye-tracking). That is because the majority of current eye-tracking technologies lose accuracy when tracking participants' attention allocation during their navigation and are difficult to apply to tracking attention allocation status towards the virtualized stimulus. However, it is still of considerable value for researchers to develop feasible solutions to improve attention-identification accuracy, for example, by using mounted eye-tracking devices (Meißner et al., 2019) or reliable computer algorithms (Kurzhals et al., 2017). More broadly, to gain more insights into attention, some other research approaches capturing cognitive and behavioral data could also be considered for integration into the attention allocation analysis. For instance, brain activity during the attention process can be recorded by event-related potentials (Nobre et al., 2000), and biological activity can be examined by electroencephalograph (Cho et al., 2002a). Attention performance, such as the attention span, could be collected using post–experiment tests (Cho et al., 2002b), as well as self-reported measurements that uncover participants' attention dynamics, strategies and underlying psychological processes.
Figures
Summary of representative studies related to spatial attention allocation in XR
| Author and year | Scenario | Technology | Attention aspect | Attention measurement | Result |
|---|---|---|---|---|---|
| None-shopping context | |||||
| Mestre et al. (2011) |
|
|
|
|
|
| Hillaire et al. (2009) |
|
|
|
|
|
| Berton et al. (2019) |
|
|
|
|
|
| Biocca et al. (2006) |
|
|
|
|
|
| Medenica et al. (2011) |
|
|
|
|
|
| Dixon et al. (2014) |
|
|
|
|
|
| Huang et al. (2019) |
|
|
|
|
|
| Shopping context | |||||
| Pfeiffer et al. (2020) |
|
|
|
|
|
| Yang et al. (2020) |
|
|
|
|
|
| Wychgel (2020) |
|
|
|
|
|
Source(s): Author's own creation/work
Descriptive results of attention indicators
| AR | VR | Attentional visit indicator | Attentional duration indicator | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Attentional visit shift | Attentional visit variation | Attentional duration shift | Attentional duration variation | ||||||
| M | SD | M | SD | M | SD | M | SD | ||
| No | No (n = 41) | 0.800 | 0.073 | 0.308 | 0.159 | 0.946 | 0.042 | 0.305 | 0.142 |
| Yes (n = 38) | 0.734 | 0.075 | 0.347 | 0.184 | 0.924 | 0.054 | 0.335 | 0.205 | |
| Yes | No (n = 41) | 0.755 | 0.078 | 0.322 | 0.194 | 0.943 | 0.057 | 0.401 | 0.230 |
| Yes (n = 39) | 0.754 | 0.077 | 0.378 | 0.175 | 0.908 | 0.066 | 0.396 | 0.214 | |
Note(s): M = Mean value; SD = Standard deviation
Source(s): Author's own creation/work
Tests of between-subject effects
| Factor | Attentional visit indicator | Attentional duration indicator | ||||
|---|---|---|---|---|---|---|
| Attentional visit shift | Attentional visit variation | Attentional duration shift | Attentional duration variation | |||
| VR | Yes (n = 77) | M | 0.744 | 0.363 | 0.916 | 0.366 |
| SD | 0.076 | 0.179 | 0.060 | 0.210 | ||
| No (n = 82) | M | 0.777 | 0.315 | 0.945 | 0.353 | |
| SD | 0.079 | 0.176 | 0.050 | 0.196 | ||
| F (1, 155) | 8.248** | 2.760 | 11.824** | 0.071 | ||
| p | 0.005 | 0.099 | 0.001 | 0.790 | ||
| Partial η2 | 0.052 | 0.018 | 0.074 | 0.000 | ||
| AR | Yes (n = 80) | M | 0.754 | 0.349 | 0.926 | 0.399 |
| SD | 0.077 | 0.186 | 0.063 | 0.221 | ||
| No (n = 79) | M | 0.768 | 0.327 | 0.935 | 0.320 | |
| SD | 0.081 | 0.171 | 0.049 | 0.174 | ||
| F (1, 155) | 0.905 | 0.379 | 1.576 | 3.961* | ||
| p | 0.343 | 0.539 | 0.211 | 0.048 | ||
| Partial η2 | 0.006 | 0.003 | 0.010 | 0.038 | ||
| Interaction effect: VR × AR | ||||||
| F (1, 155) | 9.558** | 0.009 | 0.204 | 0.435 | ||
| p | 0.002 | 0.923 | 0.653 | 0.510 | ||
| Partial η2 | 0.060 | 0.000 | 0.001 | 0.003 | ||
Note(s): M = Mean value; SD = Standard Deviation; *p < 0.05; **p < 0.01; ***p < 0.001
Source(s): Author's own creation/work
Pairwise comparisons on attentional visit shift
| Dependent variable: attentional visit shift | |||||||
|---|---|---|---|---|---|---|---|
| (I) | (J) | Mean difference (I-J) | Standard error | p | 95% Confidence interval | ||
| Lower | Upper | ||||||
| non-VR | non-AR | AR | 0.045** | 0.017 | 0.008 | 0.012 | 0.078 |
| VR | non-AR | AR | −0.020 | 0.017 | 0.249 | −0.054 | 0.014 |
| non-AR | non-VR | VR | 0.065*** | 0.017 | <0.001 | 0.032 | 0.099 |
| AR | non-VR | VR | 0.001 | 0.017 | 0.963 | −0.033 | 0.034 |
Note(s): *p < 0.05; **p < 0.01; ***p < 0.001
Source(s): Author's own creation/work
Summary of XR-led biases
| Technology | Spatial attention mechanism | Outcome | Effect | ||||
|---|---|---|---|---|---|---|---|
| VR | Cognition resource competition | Top-down | Increases attentional visit frequency on non-AOIs by leading goals shifting | Attentional visit | Shift | ↓ | Inattention effect |
| Bottom-up | – | Variation | – | – | |||
| Cognitive load | Germane | Increases germane cognitive load by increasing episodic memory | Attentional duration | Shift | ↓ | Acceleration effect | |
| Extraneous | – | Variation | – | – | |||
| AR | Cognition resource competition | Top-down | – | Attentional visit | Shift | ↓ (applied without VR) | Inattention effect |
| Bottom-up | Decreases attentional visit frequency on AOIs by bringing in exogenous cues | Variation | – | – | |||
| Cognitive load | Germane | Increases germane cognitive load by increasing episodic memory | Attentional duration | Shift | – | – | |
| Extraneous | Increases extraneous cognitive load by increasing information bandwidth | Variation | ↑ | Imbalance effect | |||
Source(s): Author's own creation/work
Coding scheme and data collection of gaze behaviors
| Behavior code | Description | Quantitative data collected | |
|---|---|---|---|
| Event-related data | Time-related data | ||
| AOI-Left | The participant’s head directed at the shelf located on the left-side wall | visitAOI-left | durationAOI-left |
| AOI-Middle | The participant’s head directed at the shelf located on the middle wall | visitAOI-middle | durationAOI-middle |
| AOI-Right | The participant’s head directed at the shelf located on the right-side wall | visitAOI-right | durationAOI-right |
| Non-AOI | The participant’s head directed at locations other than the shelves | visitnon-AOI | durationnon-AOI |
Source(s): Author's own creation/work
Series of spatial attention allocation indicators
| Shift-type indicator | Variation-type indicator | |
|---|---|---|
| Indicator based on event-related data | Attentional visit shift: the total number of attentional visits towards all AOIs divided by the total attentional visit numbers within each case | Attentional visit variation: the variation coefficient of the attentional visit across different AOIs within each case |
| Indicator based on time-related data | Attentional duration shift: the sum of time duration of all attentional visits towards all AOIs divided by the total attentional visit duration within each case | Attentional duration variation: the variation coefficient of the attentional duration across different AOIs within each case |
Source(s): Author's own creation/work
Descriptive statistics for control variables
| Control variable | Case number | Min | Max | M | SD | Kurtosis | Skewness |
|---|---|---|---|---|---|---|---|
| Gender | 159 | 1 | 2 | 1.46 | 0.500 | 0.166 | −1.998 |
| Age | 159 | 2 | 10 | 3.31 | 1.032 | 2.381 | 11.344 |
| Income | 159 | 1 | 7 | 1.67 | 1.146 | 2.167 | 4.893 |
| Education | 159 | 1 | 3 | 1.50 | 0.615 | 0.839 | −0.285 |
| VR experience | 159 | 1 | 6 | 1.94 | 1.048 | 1.566 | 2.776 |
Note(s): Min = Minimum value; Max = Maximum value; M = Mean value; SD = Standard deviation. Gender was considered as a binary variable: 1 = male, 2 = female. Age was measured from 1 = less than 15 years old, 2 = 15–19 years old, 3 = 20–24 years old, 4 = 25–29 years old, 5 = 30–34 years old, 6 = 35–39 years old, 7 = 40–44 years old, 8 = 45–49 years old, 9 = 50–54 years old, 10 = 55–59 years old, to 11 = 60 years old or above. Income (pre-tax) was measured from 1 = 0–499 euro, 2 = 500–999 euro to 9 = 4,000 euro or more. Education was measured from 1 = bachelor student, 2 = master student and 3 = PhD student. VR experience was measured from 1 = never used VR, 2 = rarely used VR, 3 = occasionally used VR, 4 = sometimes used VR, 5 = frequently used VR, 6 = usually used VR, to 7 = used VR everyday
Source(s): Author's own creation/work
One-way ANOVA results for control variables
| Sum of squares | Degree of freedom | Mean square | F | p | ||
|---|---|---|---|---|---|---|
| Gender | Between groups | 0.852 | 3 | 0.284 | 1.140 | 0.335 |
| Within groups | 38.632 | 155 | 0.249 | |||
| Total | 39.484 | 158 | ||||
| Age | Between groups | 3.595 | 3 | 1.198 | 1.128 | 0.340 |
| Within groups | 164.681 | 155 | 1.062 | |||
| Total | 168.277 | 158 | ||||
| Income | Between groups | 7.788 | 3 | 2.596 | 2.017 | 0.114 |
| Within groups | 199.545 | 155 | 1.287 | |||
| Total | 207.333 | 158 | ||||
| Education | Between groups | 1.269 | 3 | 0.423 | 1.121 | 0.342 |
| Within groups | 58.479 | 155 | 0.377 | |||
| Total | 59.748 | 158 | ||||
| VR experience | Between groups | 6.600 | 3 | 2.200 | 2.045 | 0.110 |
| Within groups | 166.771 | 155 | 1.076 | |||
| Total | 173.371 | 158 | ||||
Source(s): Author's own creation/work
Notes
Any participant who felt a need to readjust their XR devices during the experiment was offered extra help by the research assistant; the timing was suspended until the readjustment was finished.
We achieved this merge by rewinding the video to the moment before the short event occurred and annotating the event occurring after this event.
Conflicts of interest: No conflicts of interest have been declared.
Appendix
References
Albert, A., Hallowell, M.R., Kleiner, B., Chen, A. and Golparvar-Fard, M. (2014), “Enhancing construction hazard recognition with high-fidelity augmented virtuality”, Journal of Construction Engineering and Management, Vol. 140 No. 7, 04014024.
Alcañiz Raya, M., Bigné, E. and Guixeres, J. (2019), “Virtual reality in marketing: a framework, review, and research agenda”, Frontiers in Psychology, Vol. 10, 1530.
Bakeman, R. and Quera, V. (2011), Sequential Analysis and Observational Methods for the Behavioral Sciences, Cambridge University Press, Cambridge.
Bartlett, M.L., Gwinn, O.S., Thomas, N.A. and Nicholls, M.E. (2020), “Cognitive load exacerbates rightward biases during computer maze navigation”, Brain and Cognition, Vol. 140, 105547.
Berg, L.P. and Vance, J.M. (2017), “Industry use of virtual reality in product design and manufacturing: a survey”, Virtual Reality, Vol. 21, pp. 1-17.
Berton, F., Olivier, A.H., Bruneau, J., Hoyet, L. and Pettré, J. (2019), “Studying gaze behaviour during collision avoidance with a virtual walker: influence of the virtual reality setup”, In Teather, R., Itoh, Y., Gabbard, J., Argelaguet, F., Olivier, A.-H. and Keefe, D. (Eds), VR 2019: IEEE Conference on Virtual Reality and 3D User Interfaces, IEEE, pp. 718-726.
Binetti, N., Cheng, T., Mareschal, I., Brumby, D., Julier, S. and Bianchi-Berthouze, N. (2019), “Assumptions about the positioning of virtual stimuli affect gaze direction estimates during augmented reality based interactions”, Scientific Reports, Vol. 9 No. 1, pp. 1-9.
Biocca, F., Tang, A., Owen, C. and Xiao, F. (2006), “Attention funnel: omnidirectional 3D cursor for mobile augmented reality platforms”, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, ACM, pp. 1115-1122.
Bonetti, F., Warnaby, G. and Quinn, L. (2018), “Augmented reality and virtual reality in physical and online retailing: a review, synthesis and research agenda”, in Jung, T.H. (Eds), Augmented Reality and Virtual Reality, pp. 119-132.
Briand, K.A. (1998), “Feature integration and spatial attention: more evidence of a dissociation between endogenous and exogenous orienting”, Journal of Experimental Psychology: Human Perception and Performance, Vol. 24 No. 4, pp. 1243-1256.
Brooks, F.P. (1999), “What's real about virtual reality?”, IEEE Computer Graphics and Applications, Vol. 19 No. 6, pp. 16-27.
Carlson, P., Kirpes, C., Pavlik, R.A., Vance, J.M., Yin, L., Scott-Cooper, T. and Lambert, T. (2011), “Comparison of single-wall versus multi-wall immersive environments to support a virtual shopping experience”, Proceedings of ASME 2011 World Conference on Innovative Virtual Reality, pp. 287-291.
Chandon, P., Hutchinson, J.W., Bradlow, E.T. and Young, S.H. (2009), “Does in-store marketing work? Effects of the number and position of shelf facings on brand attention and evaluation at the point of purchase”, Journal of Marketing, Vol. 73 No. 6, pp. 1-17.
Chen, W., Ladeveze, N., Clavel, C., Mestre, D. and Bourdot, P. (2015), “User cohabitation in multi-stereoscopic immersive virtual environment for individual navigation tasks”, In Höllerer, T., Interrante, V., Lecuyer, A. and Swan, J.E. II (Eds), VR 2015: IEEE Conference on Virtual Reality, IEEE, pp. 47-54.
Cho, B.H., Lee, J.M., Ku, J.H., Jang, D.P., Kim, J.S., Kim, I.Y., Lee, J.H. and Kim, S.I. (2002a), “Attention enhancement system using virtual reality and EEG biofeedback”, in B. Loftin B., Chen, J.X., Rizzo, S., Goebel, M. and Hirose, M. (Eds), VR 2002: IEEE Conference on Virtual Reality, IEEE, pp. 156-163.
Cho, B.H., Ku, J., Jang, D.P., Kim, S., Lee, Y.H., Kim, I.Y., Lee, J.H. and Kim, S.I. (2002b), “The effect of virtual reality cognitive training for attention enhancement”, CyberPsychology and Behavior, Vol. 5 No. 2, pp. 129-137.
Clement, J. (2007), “Visual influence on in-store buying decisions: an eye-track experiment on the visual influence of packaging design”, Journal of Marketing Management, Vol. 23 Nos 9-10, pp. 917-928.
Cohen, A. and Shoup, R. (2000), “Response selection processes for conjunctive targets”, Journal of Experimental Psychology: Human Perception and Performance, Vol. 26 No. 1, pp. 391-411.
Cuomo, M.T., Tortora, D., Festa, G., Ceruti, F. and Metallo, G. (2020), “Managing omni-customer brand experience via augmented reality”, Qualitative Market Research, Vol. 23 No. 3, pp. 427-445.
Desimone, R. and Duncan, J. (1995), “Neural mechanisms of selective visual attention”, Annual Review of Neuroscience, Vol. 18 No. 1, pp. 193-222.
Dickinson, C.A. and Intraub, H. (2009), “Spatial asymmetries in viewing and remembering scenes: consequences of an attentional bias?”, Attention, Perception, and Psychophysics, Vol. 71 No. 6, pp. 1251-1262.
Dixon, B.J., Daly, M.J., Chan, H.H., Vescan, A., Witterick, I.J. and Irish, J.C. (2014), “Inattentional blindness increased with augmented reality surgical navigation”, American Journal of Rhinology and Allergy, Vol. 28 No. 5, pp. 433-437.
Ekhtiari, H. and Paulus, M. (2016), Neuroscience for Addiction Medicine: From Prevention to Rehabilitation-Methods and Interventions, Elsevier, Amsterdam.
Friard, O. and Gamba, M. (2016), “BORIS: a free, versatile open-source event-logging software for video/audio coding and live observations”, Methods in Ecology and Evolution, Vol. 7 No. 11, pp. 1325-1330.
Goldsmith, M. and Yeari, M. (2003), “Modulation of object-based attention by spatial focus under endogenous and exogenous orienting”, Journal of Experimental Psychology: Human Perception and Performance, Vol. 29 No. 5, pp. 897-918.
Grealy, M.A., Johnson, D.A. and Rushton, S.K. (1999), “Improving cognitive function after brain injury: the use of exercise and virtual reality”, Archives of Physical Medicine and Rehabilitation, Vol. 80 No. 6, pp. 661-667.
Hartley, A.A. (1993), “Evidence for the selective preservation of spatial selective attention in old age”, Psychology and Aging, Vol. 8 No. 3, pp. 371-379.
Hillaire, S., Lécuyer, A., Breton, G. and Corte, T.R. (2009), “Gaze behavior and visual attention model when turning in virtual environments” in Spencer, S.N. (Ed.), VRST 2009: The 16th ACM Symposium on Virtual Reality Software and Technology, ACM, pp. 43-50.
Hollebeek, L.D., Clark, M.K., Andreassen, T.W., Sigurdsson, V. and Smith, D. (2020), “Virtual reality through the customer journey: framework and propositions”, Journal of Retailing and Consumer Services, Vol. 55, 102056.
Horvitz, E., Kadie, C., Paek, T. and Hovel, D. (2003), “Models of attention in computing and communication: from principles to applications”, Communications of the ACM, Vol. 46 No. 3, pp. 52-59.
Hou, L., Wang, X., Bernold, L. and Love, P.E. (2013), “Using animated augmented reality to cognitively guide assembly”, Journal of Computing in Civil Engineering, Vol. 27 No. 5, pp. 439-451.
Huang, Y.T. (2018), “The female gaze: content composition and slot position in personalized banner ads, and how they influence visual attention in online shoppers”, Computers in Human Behavior, Vol. 82, pp. 1-15.
Huang, K.T., Ball, C., Francis, J., Ratan, R., Boumis, J. and Fordham, J. (2019), “Augmented versus virtual reality in education: an exploratory study examining science knowledge retention when using augmented reality/virtual reality mobile applications”, Cyberpsychology, Behavior, and Social Networking, Vol. 22 No. 2, pp. 105-110.
Hughes, C.E., Stapleton, C.B., Hughes, D.E. and Smith, E.M. (2005), “Mixed reality in education, entertainment, and training”, IEEE Computer Graphics and Applications, Vol. 25 No. 6, pp. 24-30.
Kaplan, A.D., Cruit, J., Endsley, M., Beers, S.M., Sawyer, B.D. and Hancock, P.A. (2020), “The effects of virtual reality, augmented reality, and mixed reality as training enhancement methods: a meta-analysis”, Human Factors, Vol. 63 No. 4, pp. 706-726.
Ketelaar, P.E., Bernritter, S.F., van Woudenberg, T.J., Rozendaal, E., Konig, R.P., Hühn, A.E., Van Gisbergen, M.S. and Janssen, L. (2018), “‘Opening’ location-based mobile ads: how openness and location congruency of location-based ads weaken negative effects of intrusiveness on brand choice”, Journal of Business Research, Vol. 91, pp. 277-285.
Kim, M.S. and Cave, K.R. (2001), “Perceptual grouping via spatial selection in a focused-attention task”, Vision Research, Vol. 41 No. 5, pp. 611-624.
Kurzhals, K., Hlawatsch, M., Seeger, C. and Weiskopf, D. (2017), “Visual analytics for mobile eye tracking”, IEEE Transactions on Visualization and Computer Graphics, Vol. 23 No. 1, pp. 301-310.
Kwok, A.O. and Koh, S.G. (2020), “COVID-19 and extended reality (XR)”, Current Issues in Tourism, Vol. 24 No. 14, pp. 1935-1940.
Lee, K.C. and Chung, N. (2008), “Empirical analysis of consumer reaction to the virtual reality shopping mall”, Computers in Human Behavior, Vol. 24 No. 1, pp. 88-104.
Lee, K.H., Chen, X., Hua, G., Hu, H. and He, X. (2018), “Stacked cross attention for image-text matching”, Proceedings of the European Conference on Computer Vision (ECCV), pp. 201-216.
Lix, L.M., Keselman, J.C. and Keselman, H.J. (1996), “Consequences of assumption violations revisited: a quantitative review of alternatives to the one-way analysis of variance F test”, Review of Educational Research, Vol. 66 No. 4, pp. 579-619.
Loftus, A.M. and Nicholls, M.E.R. (2012), “Testing the activation-orientation account of spatial attentional asymmetries using transcranial direct current stimulation”, Neuropsychologia, Vol. 50 No. 11, pp. 2573-2576.
Lutz, A., Slagter, H.A., Dunne, J.D. and Davidson, R.J. (2008), “Attention regulation and monitoring in meditation”, Trends in Cognitive Sciences, Vol. 12 No. 4, pp. 163-169.
Manis, K.T. and Choi, D. (2019), “The virtual reality hardware acceptance model (VR-HAM): extending and individuating the technology acceptance model (TAM) for virtual reality hardware”, Journal of Business Research, Vol. 100, pp. 503-513.
Maringelli, F., McCarthy, J., Steed, A., Slater, M. and Umilta, C. (2001), “Shifting Visuo-spatial attention in a virtual three-dimensional space”, Cognitive Brain Research, Vol. 10 No. 3, pp. 317-322.
Massara, F., Porcheddu, D. and Melara, R.D. (2014), “Asymmetric perception of sparse shelves in retail displays”, Journal of Retailing, Vol. 90 No. 3, pp. 321-331.
Massiceti, D., Hicks, S.L. and van Rheede, J. (2018), “Stereosonic vision: exploring visual-to-auditory sensory substitution mappings in an immersive virtual reality navigation paradigm”, PloS One, Vol. 13 No. 7, 0199389.
McCrickard, D.S. and Chewar, C.M. (2003), “Attuning notification design to user goals and attention costs”, Communications of the ACM, Vol. 46 No. 3, pp. 67-72.
McNally, R.J. (2019), “Attentional bias for threat: crisis or opportunity?”, Clinical Psychology Review, Vol. 69, pp. 4-13.
Medenica, Z., Kun, A.L., Paek, T. and Palinko, O. (2011), “Augmented reality vs street views: a driving simulator study comparing two emerging navigation aids”, Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 265-274.
Meißner, M., Pfeiffer, J., Pfeiffer, T. and Oppewal, H. (2019), “Combining virtual reality and mobile eye tracking to provide a naturalistic experimental environment for shopper research”, Journal of Business Research, Vol. 100, pp. 445-458.
Mestre, D.R., Maïano, C., Dagonneau, V. and Mercier, C.S. (2011), “Does virtual reality enhance exercise performance, enjoyment, and dissociation? An exploratory study on a stationary bike apparatus”, Presence: Teleoperators and Virtual Environments, Vol. 20 No. 1, pp. 1-14.
Mitchell, M.R. and Potenza, M.N. (2017), “Stroop, cocaine dependence, and intrinsic connectivity”, in The Neuroscience of Cocaine, Academic Press, pp. 331-339.
Naert, L., Bonato, M. and Fias, W. (2018), “Asymmetric spatial processing under cognitive load”, Frontiers in Psychology, Vol. 9, 583.
Nobre, A.C., Sebestyen, G.N. and Miniussi, C. (2000), “The dynamics of shifting visuospatial attention revealed by event-related potentials”, Neuropsychologia, Vol. 38 No. 7, pp. 964-974.
Olshausen, B.A., Anderson, C.H. and Van Essen, D.C. (1993), “A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information”, Journal of Neuroscience, Vol. 13 No. 11, pp. 4700-4719.
Otterbring, T., Wästlund, E., Gustafsson, A. and Shams, P. (2014), “Vision (im) possible? The effects of in-store signage on customers' visual attention”, Journal of Retailing and Consumer Services, Vol. 21 No. 5, pp. 676-684.
Otterbring, T., Wästlund, E. and Gustafsson, A. (2016), “Eye-tracking customers' visual attention in the wild: dynamic gaze behavior moderates the effect of store familiarity on navigational fluency”, Journal of Retailing and Consumer Services, Vol. 28, pp. 165-170.
Pantano, E., Rese, A. and Baier, D. (2017), “Enhancing the online decision-making process by using augmented reality: a two country comparison of youth markets”, Journal of Retailing and Consumer Services, Vol. 38, pp. 81-95.
Perron-Brault, A., Dantas, D. and Legoux, R. (2020), “Customer engagement and new product adoption in the music industry: the importance of fit between brand and new product innovativeness”, International Journal of Arts Management, Vol. 23 No. 1, pp. 44-60.
Pfeiffer, J., Pfeiffer, T., Meißner, M. and Weiß, E. (2020), “Eye-tracking-based classification of information search behavior using machine learning: evidence from experiments in physical shops and virtual reality shopping environments”, Information Systems Research, Vol. 31 No. 3, pp. 675-691.
Rappa, N.A., Ledger, S., Teo, T., Wai Wong, K., Power, B. and Hilliard, B. (2019), “The use of eye tracking technology to explore learning and performance within virtual reality and mixed reality settings: a scoping review”, Interactive Learning Environments, Vol. 32 No. 7, pp. 1338-1350.
Rayner, K. (1998), “Eye movements in reading and information processing: 20 Years of research”, Psychological Bulletin, Vol. 124 No. 3, pp. 372-422.
Rese, A., Baier, D., Geyer-Schulz, A. and Schreiber, S. (2017), “How augmented reality apps are accepted by consumers: a comparative analysis using scales and opinions”, Technological Forecasting and Social Change, Vol. 124, pp. 306-319.
Reuter-Lorenz, P.A., Kinsbourne, M. and Moscovitch, M. (1990), “Hemispheric control of spatial attention”, Brain and Cognition, Vol. 12 No. 2, pp. 240-266.
Riar, M., Xi, N., Korbel, J.J., Zarnekow, R. and Hamari, J. (2022), “Using augmented reality for shopping: a framework for AR induced consumer behavior, literature review and future agenda”, Internet Research, Vol. 33 No. 1, pp. 242-279.
Riva, G., Gaggioli, A., Gorini, A., Carelli, L., Repetto, C., Algeri, D. and Vigna, C. (2009), “Virtual reality as empowering environment for personal change: the contribution of the applied technology for neuro-psychology laboratory”, Anuario De Psicología, Vol. 40 No. 2, pp. 171-192.
Schoormans, J.P.L. and Robben, H.S.J. (1997), “The effect of new package design on product attention, categorization and evaluation”, Journal of Economic Psychology, Vol. 18 Nos 2-3, pp. 271-287.
Sears, D.O. (1986), “College sophomores in the laboratory: influences of a narrow data base on social psychology's view of human nature”, Journal of Personality and Social Psychology, Vol. 51, pp. 515-530.
Shi, R., Sharpe, L. and Abbott, M. (2019), “A meta-analysis of the relationship between anxiety and attentional control”, Clinical Psychology Review, Vol. 72, 101754.
Shiffrin, R.M. and Schneider, W. (1977), “Controlled and automatic human information processing: i. Detection, search, and attention”, Psychological Review, Vol. 84 No. 1, pp. 1-66.
Shulman, G.L., Wilson, J. and Sheehy, J.B. (1985), “Spatial determinants of the distribution of attention”, Perception and Psychophysics, Vol. 37 No. 1, pp. 59-65.
Statista (2022), “Augmented reality (AR), virtual reality (VR), and mixed reality (MR) market size worldwide in 2021 and 2028”, available at: https://www.statista.com/statistics/591181/global-augmented-virtual-reality-market-size/ (accessed 28 April 2022).
Streicher, M.C., Estes, Z. and Büttner, O.B. (2020), “Exploratory shopping: attention affects in-store exploration and unplanned purchasing”, Journal of Consumer Research, Vol. 48 No. 1, pp. 51-76.
Sweller, J. (2005), “Implications of cognitive load theory for multimedia learning”, The Cambridge Handbook of Multimedia Learning, Vol. 3 No. 2, pp. 19-30.
Sweller, J. (2011), “Cognitive load theory”, in Psychology of Learning and Motivation, Academic Press, Vol. 55, pp. 37-76.
Treue, S. (2003), “Visual attention: the where, what, how and why of saliency”, Current Opinion in Neurobiology, Vol. 13 No. 4, pp. 428-432.
Tucker, D.M., Hartry-Speiser, A., McDougal, L., Luu, P. and degrandpre, D. (1999), “Mood and spatial memory: emotion and right hemisphere contribution to spatial cognition”, Biological Psychology, Vol. 50 No. 2, pp. 103-125.
Wang, F.Y., Qin, R., Wang, X. and Hu, B. (2022), “Metasocieties in metaverse: metaeconomics and metamanagement for metaenterprises and metacities”, IEEE Transactions on Computational Social Systems, Vol. 9 No. 1, pp. 2-7.
Wolfe, J.M. (1998), “What can 1 million trials tell us about visual search?”, Psychological Science, Vol. 9 No. 1, pp. 33-39.
Wychgel, J. (2020), In-store advertising using augmented reality: the effectiveness of brand engagement through visual attention, Master’s Thesis, University of Twente.
Xi, N. and Hamari, J. (2021), “Shopping in virtual reality: a literature review and future agenda”, Journal of Business Research, Vol. 134, pp. 37-58.
Xi, N., Chen, J., Gama, F., Riar, M. and Hamari, J. (2022), “The challenges of entering the metaverse: an experiment on the effect of extended reality on workload”, Information Systems Frontiers, Vol. 25, pp. 659-680.
Yang, S., Carlson, J.R. and Chen, S. (2020), “How augmented reality affects advertising effectiveness: the mediating effects of curiosity and attention toward the ad”, Journal of Retailing and Consumer Services, Vol. 54, 102020.
Yantis, S. (1993), “Stimulus-driven attentional capture”, Current Directions in Psychological Science, Vol. 2, pp. 156-161.
Yantis, S. (2000), “Goal-directed and stimulus-driven determinants of attentional control”, Attention and Performance, Vol. 18, pp. 73-103.
Yim, M.Y., Chu, S. and Sauer, P.L. (2017), “Is augmented reality technology an effective tool for e-commerce? An interactivity and vividness perspective”, Journal of Interactive Marketing, Vol. 39, pp. 89-103.
Zyda, M. (2005), “From visual simulation to virtual reality to games”, Computer, Vol. 38 No. 9, pp. 25-32.
Acknowledgements
This work has been supported by KAUTE Foundation under Grant No. 20190003 and No. 20200531; OP Ryhmän Tutkimussäätiö under Grant No. 20200040; Liikesivistysrahasto under Grant No. 180386 and No. 210301; Business Finland under Grant No. 5654/31/2018 and Grant No. 4708/31/2019; Academy of Finland under Grant No. 327241 and Grant No. 337653; Youth Foundation for Humanities and Social Sciences Research of China’s Ministry of Education under Grant No. 22YJC630007, and Research Foundation of Anhui University of Finance and Economics under Grant No. ACKYB22018. This work was carried out with the support of Centre for Immersive Visual Technologies (CIVIT) research infrastructure in Tampere University, Finland. The authors thank Filipe Gama, Henrietta Jylhä, Henry Korkeila and Shuo Yang’s contribution to the programme development, experiment design and implementation. The authors also thank all student participants from Tampere University.