Elsevier

Cortex

Volume 133, December 2020, Pages 133-148
Cortex

Special Issue “Attention & Motor Processes”: Research Report
Theory of visual attention (TVA) in action: Assessing premotor attention in simultaneous eye-hand movements

https://doi.org/10.1016/j.cortex.2020.09.020Get rights and content

Abstract

Attention shifts that precede goal-directed eye and hand movements are regarded as markers of motor target selection. Whether effectors compete for a single, shared attentional resource during simultaneous eye-hand movements or whether attentional resources can be allocated independently towards multiple target locations is controversially debated. Independent, effector-specific target selection mechanisms underlying parallel allocation of visuospatial attention to saccade and reach targets would predict an increase of the overall attention capacity with the number of active effectors. We test this hypothesis in a modified Theory of Visual Attention (TVA; Bundesen, 1990) paradigm. Participants reported briefly presented letters during eye, hand, or combined eye-hand movement preparation to centrally cued locations. Modeling the data according to TVA allowed us to assess both the overall attention capacity and the deployment of visual attention to individual locations in the visual work space. In two experiments, we show that attention is predominantly allocated to the motor targets–without pronounced competition between effectors. The parallel benefits at eye and hand targets, however, have concomitant costs at non-motor locations, and the overall attention capacity does not increase by the simultaneous recruitment of both effector systems. Moreover, premotor shifts of attention dominate over voluntary deployment of processing resources, yielding severe impairments of voluntary attention allocation. We conclude that attention shifts to multiple effector targets without mutual competition given that sufficient processing resources can be withdrawn from movement-irrelevant locations.

Introduction

When briefly confronted with multiple visual objects in a display, only a subset of the information can be processed due to limited capacities of the visual system. Attention allows the selection of the currently most relevant information in order to guide our behavior adequately (Carrasco, 2011). Over the past decades, different sources of selection bias have been identified, such as selection history, physical salience, and current goals, all of which are integrated in priority maps and determine the spatial deployment of visual attention (Awh et al., 2012). Additionally, researchers have attempted to incorporate different aspects of visual attention into a single theory of attention. One of the most powerful theories is Bundesen's (1990) theory of visual attention (TVA), which accounts for a broad range of findings from behavioral, neurophysiological, and neuropsychological studies on selective attention (Bundesen & Habekost, 2014). Based on the biased competition principle (Desimone & Duncan, 1995), TVA provides a mathematical description of parallel processing of visual objects which compete for selection into visual short-term memory (vSTM). In TVA, attentional selection is achieved by perceptually categorizing the object and storing the categorization in vSTM (for a detailed conceptual description, see Bundesen, 1990; Bundesen & Habekost, 2014). In multi-element displays, where several objects compete for conscious perception and encoding in vSTM (assumed to be limited to K different elements), successful selection is determined by the speed of object processing, referred to as processing rate: The objects that are processed first win the race for successful storage in vSTM and become available for action control. According to TVA, this rate (v(x,i)) is composed of three terms: the sensory strength (η(x,i)) of an object x belonging to the category i, the perceptual bias (βi) associated with this category, and the relative attentional weight associated with the object (wx):v(x,i)=η(x,i)βiwxzSwz

Due to the limited vSTM storage capacity (Luck & Vogel, 1997), it is necessary to prioritize certain objects for further processing. In TVA, prioritization is accomplished by two attentional mechanisms: The perceptual bias (βi) determines how an object is categorized and the assignment of attentional weights (wx) filters which objects are selected for encoding in vSTM (based on the momentary importance of attending to objects belonging to a certain category; Bundesen, 1990; Bundesen & Habekost, 2014). Larger attentional weights thus increase the rate at which an object is selected and its probability to be successfully encoded into vSTM. Because attentional weights are determined for each object in the visual work space, v(x,i) describes the processing rate of each individual object. The individual processing rates sum up to the overall rate of objects categorization, which is defined as the (attentional) processing capacity (Bundesen, 1990; see also Fig. 1b):C=xSiRv(x,i)

TVA has been applied to determine age-related changes in attention selectivity and capacity across the lifespan (McAvinue et al., 2012), to assess cognitive impairments in various patient groups (Bublak et al., 2005; Duncan et al., 1999; for a review, see; Habekost, 2015), and to measure different components of visual attention in healthy populations (Finke et al., 2005). Yet, the TVA-based assessment of visual attention has only been studied under fixation (but see Poth & Schneider, 2018 for TVA-based assessment of transsaccadic competition). Here, we aim to use the theoretical and computational framework of TVA to investigate, for the first time, whether TVA can account for attention mechanisms in relation to selection-for-action (Allport, 1987; Schneider, 1995).

It is well established that both saccades (Deubel & Schneider, 1996; Kowler et al., 1995; Montagnini & Castet, 2007) and hand movements (Baldauf et al., 2006; Deubel et al., 1998; Rolfs et al., 2013) are preceded by obligatory shifts of visual attention towards their motor targets. Some authors even assume that the very purpose of attention is action control (Allport, 1987; Neumann, 1987). Nonetheless, it is unknown how action-coupled attention shifts modify the selection of competing objects for encoding into vSTM. To approach this question, we combined a TVA paradigm with eye-hand movement tasks to investigate how motor preparation affects the deployment of visual attention over multi-element displays. Whereas the classical TVA framework focusses mainly on single-task situations, Logan and Gordon (2001) previously extended TVA to an executive control theory of visual attention (ECTVA) that can account for dual-task situations (for an application of TVA to motor-cognitive dual-task situations, see Künstler et al., 2018). Yet, to the best of our knowledge, our study is the first to evaluate premotor shifts of visual attention via the TVA framework, which offers two key advantages over conventional approaches: First, given the detailed theoretical framework of TVA, we are able to make detailed predictions on how premotor shifts of visual attention affect the selection of competing objects for successful encoding into vSTM. Second, using a multi-element display, TVA allows the assessment of both the overall attention capacity (parameter C, captured in the visual processing capacity of letters) and the allocation of attentional resources to each individual object in a display (parameter v). Thus, overcoming the limitations of traditional premotor paradigms that can measure attention only at one location at a time (see Hanning et al., 2019 for different approaches), the TVA paradigm allows for a concurrent assessment of premotor visual processing across the entire visual work space.

The advantage of being able to simultaneously evaluate perceptual benefits and costs at motor targets and movement-irrelevant locations becomes particularly evident in the investigation of attentional dynamics during the preparation of combined eye-hand movements: Looking and reaching simultaneously to spatially separate goals requires the selection of multiple movement targets. Previous studies investigating shifts of visual attention–an index of movement target selection–during simultaneous eye-hand movements, observed increased attention at both eye and hand targets (Hanning et al., 2018; Jonikaitis & Deubel, 2011; Khan et al., 2011). Whereas Khan et al. (2011) observed that the eye dominated in guiding attention during simultaneous eye-hand movements and argued in favor of a single, shared attentional resource (see also Nissens & Fiehler, 2018), other studies showed that the attention benefits at two effector targets are not impaired by the necessity to plan simultaneous eye-hand movements versus a single eye or hand movement (Hanning et al., 2018; Jonikaitis & Deubel, 2011). On the one hand, a parallel allocation of attention to multiple effector targets without competition can be explained by separate attentional resources dedicated to each individual effector. In this case, the overall attention capacity should increase with the number of active effectors. On the other hand, the eye and hand target benefits may be accompanied by a withdrawal of attentional resources from non-motor locations–in which case the overall attention capacity would not rise with the number of active effectors.

To unravel this ambiguity, we used TVA to asses visual processing across the entire visual work space, as well as the allocation of attention to the individual motor targets and movement-irrelevant locations. In Experiment 1, we assessed whether the parallel preparation of eye and hand movements to spatially separate goals enhances visual processing at both effector targets simultaneously, and whether such parallel processing benefits would be reflected in increased overall attention capacity. In Experiment 2, we investigated whether attention was withdrawn from non-motor locations in order to allocate processing resources towards the motor targets and whether these costs occurred obligatorily. We used a similar paradigm as in Experiment 1, but in addition to the perceptual and motor task, we gave participants an incentive to voluntarily deploy their attention primarily to a subset of the presented items: different colors (50% red and 50% blue) indicated whether the letters were associated with a high or low monetary reward. This enabled us to investigate the ability to voluntarily deploy attention while preparing eye-hand movement.

Altogether, our experimental design allows us to determine (a) whether attention is deployed to the eye and hand targets in parallel, (b) whether such parallel allocation leads to an increase in the overall attention capacity or is associated with concomitant costs at non-motor locations, and (c) how our goal-directed actions influence our ability to voluntarily attend elsewhere.

Section snippets

Experiment 1

In this experiment we established the TVA framework as a sensitive tool to measure action-related shifts of visual attention. Specifically, we expected action preparation to increase the attentional weights at the motor targets, reflected in increased processing rates. First, we used this approach to examine if attention is deployed in parallel to eye and hand movement targets and whether the effectors compete for attentional resources. Second, we tested whether the overall attention capacity

Experiment 2

The second experiment was designed to assess whether the increased attention deployment at motor targets is achieved by withdrawal of processing resources from non-target locations. To give participants an incentive to deploy their attention also to non-target locations, we introduced a reward manipulation. Monetary reward has been shown to modulate voluntary selective visual attention (Della Libera & Chelazzi, 2006; for a recent review see; Failing & Theeuwes, 2018), as indicated by faster

Discussion

In our novel premotor TVA paradigm, participants reported briefly presented letters while preparing eye, hand, or simultaneous eye-hand movement. To the best of our knowledge, this study is the first to apply TVA to investigate attentional dynamics during action preparation. By simultaneously assessing visual processing at motor targets and movement-irrelevant locations, we show that TVA is a sensitive and innovative tool to evaluate premotor attentional dynamics.

In TVA, objects in the visual

Conclusion

TVA-based assessments of visual attention have been widely applied in both clinical (Habekost, 2015) and basic research (Bundesen & Habekost, 2014) – but solely under fixation conditions. By combining TVA with motor tasks, we are the first to show that this framework can also be used to evaluate the dynamics of visual attention associated with motor preparation. Our study demonstrates that the commonly observed coupling of attention and goal-directed actions can be accommodated within the

Author note

The data and digital experimental materials are available on the Open Science Framework (https://osf.io/h9342/). No part of the study was pre-registered prior to the research being conducted. We report how we determined our sample size, all data exclusions, all inclusion/exclusion criteria, whether inclusion/exclusion criteria were established prior to data analysis, all manipulations, and all measures in the Methods section.

Author contributions

All authors developed the study concept and contributed to the study design. P.K. and N.M.H. collected and analyzed the Data. All authors interpreted the data. P.K. drafted the manuscript, and H.D. and N.M.H. provided critical revisions. All authors approved the final version of the manuscript for submission.

Open practices

The study in this article earned Open Materials and Open Data badges for transparent practices.

Declaration of competing interest

The authors declare no conflicts of interest.

Acknowledgements

The authors thank Marleen Haupt, Natan Napiórkowski, Christian Poth, and members of the Deubel laboratory for helpful advices and discussions. This work was supported by the Deutsche Forschungsgemeinschaft (DFG) [DE336/5-1].

References (46)

  • P. Bublak et al.

    Usability of a theory of visual attention (TVA) for parameter-based measurement of attention II: Evidence from two patients with frontal or parietal damage

    Journal of the International Neuropsychological Society: JINS

    (2005)
  • C. Bundesen

    A theory of visual attention

    Psychological Review

    (1990)
  • C. Bundesen et al.

    Theory of visual attention (TVA)

  • C. Bundesen et al.

    A neural theory of visual attention: Bridging cognition and neurophysiology

    Psychological Review

    (2005)
  • F.W. Cornelissen et al.

    The eyelink toolbox: Eye tracking with MATLAB and the Psychophysics toolbox

    Behavior Research Methods, Instruments, and Computers

    (2002)
  • C. Della Libera et al.

    Visual selective attention and the effects ofmonetary rewards

    Psychological Science

    (2006)
  • R. Desimone et al.

    Neural mechanisms of selective visual attention

    Annual Review of Neuroscience

    (1995)
  • H. Deubel

    The time course of presaccadic attention shifts

    Psychological Research

    (2008)
  • H. Deubel et al.

    Selective dorsal and ventral processing: Evidence for a common attentional mechanism in reaching and perception

    Visual Cognition

    (1998)
  • J. Duncan et al.

    Systematic analysis of deficits in visual attention

    Journal of Experimental Psychology. General

    (1999)
  • R. Engbert et al.

    Microsaccades are triggered by low retinal image slip

    Proceedings of the National Academy of Sciences of the United States of America

    (2006)
  • M. Failing et al.

    Selection history: How reward modulates selectivity of visual attention

    Psychonomic Bulletin & Review

    (2018)
  • K. Finke et al.

    Usability of a theory of visual attention (TVA) for parameter-based measurement of attention I: Evidence from normal subjects

    Journal of the International Neuropsychological Society: JINS

    (2005)
  • Cited by (0)

    View full text