Following the gold trail: Reward influences on spatial exploration in neglect

Abstract

Spatial attention is guided by the perceived salience and relevance of objects in the environment, a process considered to depend on a broad parieto-frontal cortical network. Signals arising from the limbic and nigrostriatal pathways conveying affective and motivational cues are also known to modulate visual selection, but the nature of this contribution and its relation to spatial attention remain unclear. We investigated the role of reward information in 15 patients with left hemispatial neglect and 15 control subjects playing multiple rounds of a virtual foraging game. Participants’ exploration tracked dynamically adjusted underlying reward distributions, largely unbeknownst to them. Both control and neglect participants showed typical exploration/exploitation balance, dependent on abundance or scarcity of rewards. De-reinforcing previously favored, mostly right, regions of space attenuated left space under-exploration in patients. Multiple regression analysis indicates that such reward-based training may benefit mostly patients early after lesion onset, with mild neglect and small lesions sparing subcortical regions. Our findings support the view that spatial exploration recruits heavily right hemispheric visuospatial attentional mechanisms as well as reward signals processed by basal ganglia and prefrontal cortical circuits, which serve to learn about the motivational relevance of environmental stimuli and help prioritize attention and motor response selection.

Introduction

Selective attention is a main gateway for higher-order sensory information processing. It is common to distinguish between stimulus-driven attention, causing us to orient automatically toward salient events and voluntary attention, which selects particular objects or spatial locations according to current goals and intentions. In addition to these well-studied processes, there is growing evidence that attention is influenced by learned reward associations (Chelazzi et al., 2014, Hickey et al., 2014). This is not surprising, as the brain has evolved to learn about stimuli that signal reward opportunities. Foraging animals decide to occupy patches where food is abundant and disengage from depleted ones using time-varying reward signals and they learn about reward-predictive features of their environment in the process (Stephens, 2008). Studies have begun to explore how reward history competes with stimulus-driven and goal-directed attention and have shown, for example, that previously rewarded but task-irrelevant visual items can capture attention and interfere with performances during cued-detection (Munneke, Hoppenbrouwers, & Theeuwes, 2015) or search tasks (Bourgeois, Neveu, & Vuilleumier, 2016).

Cerebral damage can lead to impairment in spatial orienting, as illustrated by the neurological syndrome of unilateral spatial neglect (Bisiach and Vallar, 2000, Heilman et al., 1993). The deficit observed in patients with neglect offers an opportunity to investigate possible interactions between reward and attention mechanisms at the brain level. Neglect arises from lesions within a broad fronto-parietal network (Corbetta & Shulman, 2011) and is characterized by a failure to report or act upon stimuli presented in contralesional space - typically the left side as a result of a right hemispheric lesion - despite intact early sensory processing (Driver & Vuilleumier, 2001). Such deficits bear upon both stimulus-driven and goal-directed attention (Bays et al., 2010, Corbetta and Shulman, 2002). Interestingly, neglect is also modulated by the emotional and motivational contents of stimuli. For instance, patients with left neglect fail to report a neutral stimulus displayed in the contralesional field when simultaneously presented with an ipsilesional one, but detection improves when left stimuli are emotionally-loaded images of frightful faces, gory scenes or spiders (Grabowska et al., 2011, Tamietto et al., 2007, Vuilleumier and Schwartz, 2001a, Vuilleumier and Schwartz, 2001b). This suggests that emotional cues undergo independent processing in intact brain structures and can somehow boost neural activity in the attention orienting network (Domínguez-Borràs, Saj, Armony, & Vuilleumier, 2012). An early report showed that detection performance of a patient with left neglect was improved by rewarding each detected target with one penny (Mesulam, 1985). In a similar vein, patients showed better performance on a paper and pencil item cancellation test when the items were images of £1.00 coins and the experimenter promised a monetary reward for each cancelled target, than when the items were images of brass buttons and there were no reward instructions (Malhotra, Soto, Li, & Russell, 2013). Although such results could merely reflect non-specific effects on motivation or arousal level, other investigators have attempted to selectively enhance the motivational value of left-sided stimuli in the context of visual search performance (Lucas et al., 2013) or forced-choice between two lateralized targets (Lecce et al., 2015). Their results indicate that neglect patients show reduced rightward attentional bias when the highest reward probability was associated with left spatial locations, thus suggesting a role of the cortical-striatal circuits linked to the dopaminergic reward system in mediating such effects.

These results suggest that when performing an attention task, reward information is processed by non-damaged structures that could indirectly and independently supply modulatory inputs into a hypothetical saliency map. Although reward-based manipulations can counteract the consequences of core attentional and awareness impairments in patients, several aspects need to be explored in more detail: whether neglect patients and normal subjects show qualitatively similar ability to learn about rewards during spatial exploration, whether similar principles govern immediate, short-range processing of reward signals and more long-term learning about reward distributions, whether reward effects take place above or below the radar of conscious awareness and whether the observed changes are context-specific or generalize to other spatial tasks. Addressing such questions could help better understanding the neurocognitive mechanisms by which reward information guides spatial exploration.

Here, we address these questions in the context of virtual foraging for hidden rewards. We asked whether patients with neglect respond to reward distributions during spatial exploration and learn to orient to high-value stimuli located in the attentional shadow of the neglected field. We used a foraging task modelled on classical paper and pencil cancelation tests in which patients must cross out or circle all or a subset of items in a large array of stimuli. Specifically, participants searched for “gold nugget” rewards hidden under 48 pictures of stones that were displayed on a touch-sensitive computer monitor (Fig. 1A). The search items varied slightly in shape and colour but contained no information about reward location. Average reward probability over the search array was set at constant p = .5 and participants were allowed to freely sample any location, in any order and at their own pace until they had touched 20 stones. This procedure was repeated seven times in a single session. The first and last runs used a spatially uniform reward probability distribution and served as initial and final reference conditions. Critically, the reward probability distributions were manipulated during the five intermediate runs using an online adaptive method. These distributions were computed so as to minimize reward probability at the most visited locations during the preceding run and maximize it at the least visited locations. This was done by computing the logistic regression fit on the horizontal spatial distribution of selected items, reversing the sign of its slope and normalizing it in order to obtain a new reward probability distribution, but with the same average p = .5, for the next run.

Thanks to this procedure, we aimed to estimate the extent to which spatial exploration by neglect patients and a matched normal control group implicitly followed the underlying reward distribution. Patients were expected to show an initial rightward tendency and therefore to be presented with a subsequent left-biased reward distribution (Fig. 1B). We predicted that low reward rates on the right side would induce a displacement of their spatial exploration toward the left side. Note that if the observed behavior matched this prediction and patients preferentially sampled the most rewarded region, they would be expected to experience both leftward and rightward shifts in the reward distribution over successive runs. Although at the group level control subjects were not expected to present a rightward bias, we predicted that individual subjects would not sample the display in a strictly homogeneous manner, some exploring more the left side and others more the right side. This would cause them to experience biased reward distributions during the intermediate runs with multiple reversals of the reward distribution in subjects closely tracking its centre of mass.