Elsevier

Cortex

Volume 122, January 2020, Pages 40-60
Cortex

Special issue: Review
Representation of shape, space, and attention in monkey cortex

https://doi.org/10.1016/j.cortex.2019.06.005Get rights and content

Abstract

Attentional deficits are core to numerous developmental, neurological, and psychiatric disorders. At the single-cell level, much knowledge has been garnered from studies of shape and spatial properties, as well as from numerous demonstrations of attentional modulation of those properties. Despite this wealth of knowledge of single-cell responses across many brain regions, little is known about how these cellular characteristics relate to population level representations and how such representations relate to behavior; in particular, how these cellular responses relate to the representation of shape, space, and attention, and how these representations differ across cortical areas and streams. Here we will emphasize the role of population coding as a missing link for connecting single-cell properties with behavior. Using a data-driven intrinsic approach to population decoding, we show that both ‘what’ and ‘where’ cortical visual streams encode shape, space, and attention, yet demonstrate striking differences in these representations. We suggest that both pathways fully process shape and space, but that differences in representation may arise due to their differing functions and input and output constraints. Moreover, differences in the effects of attention on shape and spatial population representations in the two visual streams suggest two distinct strategies: in a ventral area, attention or task demands modulate the population representations themselves (perhaps to expand or enhance one part at the expense of other parts) while in a dorsal area, at a population representation level, attention effects are weak and nearly non-existent, perhaps in order to maintain veridical representations needed for visuomotor control. We show that an intrinsic approach, as opposed to theory-driven and labeled approaches, is useful for understanding how representations develop and differ across brain regions. Most importantly, these approaches help link cellular properties more tightly with behavior, a much-needed step to better understand and interpret cellular findings and key to providing insights to improve interventions in human disorders.

Section snippets

Attention: single-cell versus population representations

Attention is a critical component of executive functions (Lyon and Krasnegor, 1996, Sereno and Bolding, 2017). Executive functions are a set of cognitive functions that are essential for goal-oriented behavior and drive our ability to adapt to a constantly changing environment. Executive function deficits lead to disproportionate impairment in function and activities of daily living (Cahn-Weiner et al., 2002, Rabinovici et al., 2015). Attention is essential in order for individuals to generate

Representations: extrinsic versus intrinsic approaches

Perceptual and cognitive representations are encoded in a collective fashion by populations of neurons and not individual neurons in isolation. Dealing with the collective behavior of neural populations leads to complex and sometimes non-intuitive properties that often require extensive quantitative methods to characterize. This has led to a widespread theoretical literature dealing with population codes, (e.g., Averbeck et al., 2006, Deneve et al., 1999, Földiák, 1993, Jazayeri and Movshon,

Cortical streams: object recognition versus spatial processing

Much early research (e.g., Brown and Schafer, 1888, Ferrier and Yeo, 1884) supports the idea of a division in visual processing between a ventral and a dorsal cortical region, which subserve object (Fig. 1) and spatial (Fig. 2) vision, respectively. More recent work also demonstrates evidence for functionally and anatomically distinct multisynaptic streams or projections from striate cortex (Ungerleider & Mishkin, 1982). These distinct visual cortical streams were first characterized as a

Cortical streams: segregation of properties or segregation of function?

Notwithstanding proliferation and identification of sub-streams, the idea remains that cortical processing of visual information from the retina is anatomically segregated into two fundamental streams, ventral versus dorsal, important for object and spatial vision, respectively. Despite this consensus, evidence has accumulated from our own and others’ work demonstrating object representations in the dorsal stream (Durand et al., 2007, Konen and Kastner, 2008, Lehky and Sereno, 2007, Murata

Intrinsic advantages: data-driven categorization and discrimination

Intrinsic methods, using data-driven (agnostic) techniques such as MDS, are able to reveal relationships among individual neurons, such as category membership, without any a priori knowledge or assumptions about the properties of individual neurons beyond their firings (for example, without time-consuming measurements of neural tuning curves across a population) (Lehky et al., 2013). For fMRI data, the same data analysis techniques can be used, substituting voxels for neurons. Intrinsic

Intrinsic advantages: data-driven cross-stream comparison of shape and space encoding

In our own work in monkeys at a cellular or fMRI level, we report many mixed findings with shape selectivity in dorsal stream neurons (Fig. 4, Fig. 5) (Peng et al., 2008, Sereno and Maunsell, 1998, Sereno et al., 2002) and retinal and eye position spatial selectivities in ventral stream neurons (Fig. 6) (Lehky et al., 2008, Sereno and Lehky, 2011a, Sereno et al., 2014). By using an intrinsic approach on the population data collected under identical conditions, we directly compared cortical

Intrinsic modeling: elucidation of key physiological characteristics

By using an identical intrinsic approach with model neurons, we can explore and elucidate how single cell properties influence these population representations. Specifically, we can search for those key physiological characteristics that can explain the differences in encoding we see across cortical regions or instead identify invariances in these population representations (those changes in physiological properties that do not affect the encoding representation). We highlight here one example

Effects of attention on population representations across cortical streams

Neurophysiological effects of attention appear to be ubiquitous in both the ventral and dorsal visual streams (Chelazzi et al., 2011, Colby and Goldberg, 1999, Desimone and Duncan, 1995, Maunsell, 2015, Reynolds and Chelazzi, 2004). In the ventral stream, this includes attentional effects in V4 (McAdams and Maunsell, 1999, Moran and Desimone, 1985, Nandy et al., 2017) as well as inferotemporal cortex (e.g., Chelazzi et al., 1998, Zhang et al., 2011). In the dorsal stream, attentional effects

Future directions: intrinsic modeling of attention

In order to better understand these attentional differences (i.e., population coding differences at a representational level due to an attention manipulation), and more specifically, the effects of various cellular attentional modulations on the response distance or discriminability of stimulus items in a high-dimensional neural representation, a model with synthetic responses based on prior physiological data would be helpful. In our own work, we have previously had much success applying MDS

Summary and implications

Despite a wealth of knowledge about single-cell response for shape, space, and attention across many brain regions, little is understood about how these properties relate to differences in population representations across cortical regions, important for behavior and intervention in human disease. Recent work has blurred distinctions between properties in the ‘what’ (ventral) and ‘where’ (dorsal) cortical visual streams, with findings demonstrating single-cell shape, space, and attention

Conclusion

Although much has been learned about the detailed response properties at the single-cell level of many regions of the brain, a deeper understanding of how these properties collectively lead to different functions at the population level in various cortical regions and how they relate to behaviors such as attention remains elusive. Without a clearer understanding of behaviors such as attention, current interventions remain rather haphazard, and optimal interventions in human disease stay a

Acknowledgments

This research was supported in part by Purdue University start-up funds (ABS) and NIH R01 MH63340.

References (162)

  • S. Kastner et al.

    A brief comparative review of primate posterior parietal cortex: A novel hypothesis on the human toolmaker

    Neuropsychologia

    (2017)
  • D.J. Kravitz et al.

    The ventral visual pathway: An expanded neural framework for the processing of object quality

    Trends in Cognitive Sciences

    (2013)
  • N. Kriegeskorte et al.

    Matching categorical object representations in inferior temporal cortex of man and monkey

    Neuron

    (2008)
  • S. Levin

    Frontal lobe dysfunctions in schizophrenia--II. Impairments of psychological and brain functions

    Journal of Psychiatric Research

    (1984)
  • J. Martínez-Trujillo et al.

    Attentional modulation strength in cortical area MT depends on stimulus contrast

    Neuron

    (2002)
  • J.C. Martinez-Trujillo et al.

    Feature-based attention increases the selectivity of population responses in primate visual cortex

    Current Biology

    (2004)
  • J.H. Maunsell et al.

    Feature-based attention in visual cortex

    Trends in Neurosciences

    (2006)
  • M. Mishkin et al.

    Object vision and spatial vision: Two cortical pathways

    Trends in Neurosciences

    (1983)
  • J.F. Mitchell et al.

    Differential attention-dependent response modulation across cell classes in macaque visual area V4

    Neuron

    (2007)
  • J.F. Mitchell et al.

    Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4

    Neuron

    (2009)
  • T.N. Aflalo et al.

    Organization of the macaque extrastriate visual cortex re-examined using the principle of spatial continuity of function

    Journal of Neurophysiology

    (2011)
  • G. Aguirre

    Number of neurons in a voxel Retrieved Dec. 1, 2018, from

    (2012)
  • A. Allport

    Selection for action: Some behavioral and neurophysiological considerations of attention and action

  • F. Arcizet et al.

    Activity in LIP, but not V4, matches performance when attention is spread

    Cerebral Cortex

    (2018)
  • B.B. Averbeck et al.

    Neural correlations, population coding and computation

    Nature Reviews Neuroscience

    (2006)
  • C. Baldassi et al.

    Shape similarity, better than semantic membership, accounts for the structure of visual object representations in a population of monkey inferotemporal neurons

    PLoS Computational Biology

    (2013)
  • N.P. Bichot et al.

    Parallel and serial neural mechanisms for visual search in macaque area V4

    Science

    (2005)
  • J.W. Bisley et al.

    Neuronal activity in the lateral intraparietal area and spatial attention

    Science

    (2003)
  • M.C.A. Booth et al.

    View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex

    Cerebral Cortex

    (1998)
  • E. Borra et al.

    Cortical connections of the macaque anterior intraparietal (AIP) area

    Cerebral Cortex

    (2008)
  • S. Bracci et al.

    Dissociations and associations between shape and category representations in the two visual pathways

    Journal of Neuroscience

    (2016)
  • F. Bremmer

    Eye position effects in macaque area V4

    Neuroreport

    (2000)
  • R. Brette

    Is coding a relevant metaphor for the brain?

    Behavioral and Brain Sciences

    (2019)
  • S. Brown et al.

    An investigation into the functions of the occipital and temporal lobes of the monkey's brain

    Philosophical Transactions of the Royal Society of London B

    (1888)
  • D.A. Cahn-Weiner et al.

    Tests of executive function predict instrumental activities of daily living in community-dwelling older individuals

    Applied Neuropsychology

    (2002)
  • L. Chang et al.

    The code for facial identity in the primate brain

    Cell

    (2017)
  • L. Chelazzi et al.

    Neural basis of visual selective attention

    Wiley Interdisciplinary Reviews Cognitive Science

    (2011)
  • L. Chelazzi et al.

    Responses of neurons in inferior temporal cortex during memory-guided visual search

    Journal of Neurophysiology

    (1998)
  • M.R. Cohen et al.

    Attention improves performance primarily by reducing interneuronal correlations

    Nature Neuroscience

    (2009)
  • C.L. Colby et al.

    Visual, presaccadic, and cognitive activation of single neurons in monkey lateral intraparietal area

    Journal of Neurophysiology

    (1996)
  • C.L. Colby et al.

    Space and attention in parietal cortex

    Annual Review of Neuroscience

    (1999)
  • A.C. Connolly et al.

    The representation of biological classes in the human brain

    Journal of Neuroscience

    (2012)
  • C. Cortes et al.

    Support-vector networks

    Machine Learning

    (1995)
  • T. Çukur et al.

    Attention during natural vision warps semantic representation across the human brain

    Nature Neuroscience

    (2013)
  • G. Deco et al.

    The role of early visual cortex in visual integration: A neural model of recurrent interaction

    European Journal of Neuroscience

    (2004)
  • S. Deneve et al.

    Reading population codes: A neural implementation of ideal observers

    Nature Neuroscience

    (1999)
  • R. Desimone

    Visual attention mediated by biased competition in extrastriate visual cortex

    Philosophical Transactions of the Royal Society of London Series B Biological Sciences

    (1998)
  • R. Desimone et al.

    Stimulus-selective properties of inferior temporal neurons in the macaque

    Journal of Neuroscience

    (1984)
  • R. Desimone et al.

    Neural mechanisms of selective visual attention

    Annual Review of Neuroscience

    (1995)
  • C. Distler et al.

    Cortical connections of inferior temporal area TEO in macaque monkeys

    Journal of Comparative Neurology

    (1993)
  • Cited by (7)

    • Reduced 2D form coherence and 3D structure from motion sensitivity in developmental dyscalculia

      2022, Neuropsychologia
      Citation Excerpt :

      In general, however, considering shape processing to be an exclusive property of the ventral visual pathway appears to be an oversimplification. Neurons in the lateral parietal area (LIP) for example, show selectivity for simple shapes of different forms (e.g. Sereno and Maunsell, 1998; Sereno et al., 2020), and one fMRI study found preferential activation to line segments forming coherent shapes compared to non-coherent ones in several areas along the ventral and the dorsal stream, including the intraparietal sulcus (Braddick et al., 2000). Other imaging and neuropsychological studies pointed at the involvement of the parietal cortex in perception of more complex objects, Gestalt perception, or in binding shape and surface details together (Humphreys, 2003; Eger et al., 2007; Himmelbach et al., 2009; Huberle and Karnath, 2012; Zaretskaya et al., 2013; Rennig et al., 2015).

    • The role of temporal cortex in the control of attention

      2022, Current Research in Neurobiology
    View all citing articles on Scopus
    View full text