Twos in human visual perception

doi:10.1016/j.cortex.2020.06.005

Cortex

Volume 132, November 2020, Pages 473-478

https://doi.org/10.1016/j.cortex.2020.06.005 Get rights and content

Abstract

Human vision serves the social function of detecting and discriminating with high efficiency conspecifics and other animals. The social world is made of social entities as much as the relations between those entities. Recent work demonstrates that vision encodes visuo-spatial relations between bodies with the same efficiency and high specialization of face/body perception. Specifically, perception of face-to-face (vs. non-facing) bodies evokes effects compatible with the most robust markers of face-specificity such as the behavioral inversion effect and increased activity in selective visual areas. Another set of results suggests that face-to-face bodies are processed as a grouped unit, analogously to facial features in a face. The facing dyad in the visual cortex may be the earliest rudimentary representation of social interaction.

Section snippets

Sociality shapes perception

A decade ago, vision scientist, Ken Nakayama, announced that vision science was going social (Nakayama, 2011). The social function of vision appears clear from the highly specialized mechanisms for face and body perception. Embracing the idea that vision has been massively shaped by the requirements of social life, Nakayama anticipated the discovery of perceptual adaptations for other aspects of the social world.

Faces and bodies are the physical actors of social life; but social life is only

Visual tuning for facing dyads

A face is attended to, detected and discriminated in terms of identity much better than an inverted face (e.g., Bruyer, 2011; Jiang, Costello, & He, 2007; Purcell & Stewart, 1988; Rezlescu, Susilo, Wilmer, & Caramazza, 2017). The cost of inversion, or face inversion effect (FIE), is higher for faces than for other familiar objects. The FIE denotes neural tuning to the face configuration, which makes perception particularly efficient, though particularly susceptible to spatial perturbation.

Attentional advantage of facing dyads

The interpretation of the inversion effect as an index of visual tuning to face and body configurations meshes well with the ability of those stimuli to recruit attention more strongly than other objects, as documented with, among others, visual search and change detection tasks (Cohen, Alvarez, Nakayama, & Konkle, 2017; Downing, Bray, Rogers, & Childs, 2004; Hershler & Hochstein, 2005; New, Cosmides, & Tooby, 2007). Recently, an attentional advantage has been reported for facing body dyads,

Neural tuning and representational sharpening

Compared with their scrambled version or other familiar objects, faces and bodies increase activity in specific areas of the visual cortex. A recent functional magnetic resonance imaging study shows that face- and body-selective cortices also respond to facing bodies more strongly than to identical but not facing bodies (Abassi & Papeo, 2020, Fig. 1). The same brain areas do not distinguish between interacting and noninteracting people based on contextual cues (e.g., accessories or clothing) or

Grouping of bodies: effects of access versus encoding

Faster/better stimulus detection and recognition of stimuli encompassing multiple parts (or objects) has often been linked to perceptual grouping, that is the representation of parts as a structure unit, or a group. Perceptual grouping is invoked across different levels of vision and types of stimuli: It applies to dots and lines in gestalt perception, as well as to facial features in a face, and parts in a body –although the relationship between gestalt-like grouping and configural processing,

Multiple-body versus multiple-object perception

Markers of face specificity, such as attentional advantage, inversion effect, and increased activity in visual areas specifically linked to configural processing, suggest analogous visual specificity for facing dyads. But, are multiple bodies (or dyads, at least) special relative to other multiple-object sets?

Object pairs in regular, meaningful configurations (e.g., a lamp above a table) are recognized particularly efficiently, suggesting that they too might be processed as a grouped unit (

Conclusions

Research on multiple-body perception suggests that spatial relations among objects matter in visual perception, so much that a body is represented in profoundly different ways depending on whether or not it faces toward another, and is reciprocated. If it does, a new attentional/perceptual unit is formed, larger and more complex than a single object.

This research defines a new class of stimuli that, I propose, can be better described and understood by analogy with faces. Like faces, facing

Acknowledgments

The author is grateful to Jean-Rémy Hochmann and Alfonso Caramazza for valuable discussion on an early version of this article. This work was supported by the European Research Council (Horizon2020, Project THEMPO, Starting Grant 758473).

References (49)

A. Baeck et al.
The distributed representation of random and meaningful object pairs in human occipitotemporal cortex: The weighted average as a general rule
Neuroimage
(2013)
P.E. Downing et al.
Bodies capture attention when nothing is expected
Cognition
(2004)
A. Hafri et al.
Encoding of event roles from visual scenes is rapid, spontaneous, and interacts with higher-level visual processing
Cognition
(2018)
O. Hershler et al.
At first sight: A high-level pop out effect for faces
Vision Research
(2005)
D. Kaiser et al.
Object vision in a structured world
Trends Cogn Sci
(2019)
D. Maurer et al.
The many faces of configural processing
Trends Cogn Sci
(2002)
S. Quadflieg et al.
The neural basis of perceiving person interactions
Cortex; a Journal Devoted To the Study of the Nervous System and Behavior
(2015)
K.L. Roberts et al.
Action relationships concatenate representations of separate objects in the ventral visual system
Neuroimage
(2010)
T. Stein et al.
Privileged detection of conspecifics: Evidence from inversion effects during continuous flash suppression
Cognition
(2012)
T. Vestner et al.
Why are social interactions found quickly in visual search tasks?
Cognition
(2020)

J. Walbrin et al.

Neural responses to visually observed social interactions

Neuropsychologia

(2018)

M.F. Wurm et al.

Lateral occipitotemporal cortex encodes perceptual components of social actions rather than abstract representations of sociality

Neuroimage

(2019)

E. Abassi et al.

The representation of two-body shapes in the human visual cortex

The Journal of Neuroscience: the Official Journal of the Society for Neuroscience

(2020)

S. Baron-Cohen

How to build a baby that can read minds: Cognitive mechanisms in mindreading

The maladapted mind: Classic readings in evolutionary psychopathology

(1997)

E. Birmingham et al.

Human social attention

Annals of the New York Academy of Sciences

(2009)

T. Brandman et al.

Interaction between scene and object processing revealed by human fMRI and MEG decoding

The Journal of Neuroscience: the Official Journal of the Society for Neuroscience

(2017)

R. Bruyer

Configural face processing: A meta-analytic survey

Perception

(2011)

S. Carey

Becoming a face expert

Philosophical Transactions of the Royal Society of London B Biological Scienceshilos Trans R Soc Lond B Biol Sci

(1992)

S. Carey

Concept innateness, concept continuity, and bootstrapping

The Behavioral and Brain Sciences

(2011)

M.A. Cohen et al.

Visual search for object categories is predicted by the representational architecture of high-level visual cortex

Journal of Neurophysiology

(2017)

S.M. Collishaw et al.

Featural and configurational processes in the recognition of faces of different familiarity

Perception

(2000)

X. Ding et al.

Two equals one: two human actions during social interaction are grouped as one unit in working memory

Psychological Science

(2017)

C. Firestone et al.

Cognition does not affect perception: Evaluating the evidence for “top-down” effects

The Behavioral and Brain Sciences

(2016)

A. Hafri et al.

Getting the gist of events: Recognition of two-participant actions from brief displays

Journal of Experimental Psychology. General

(2013)

Cited by (44)

Moral thin-slicing: Forming moral impressions from a brief glance
2024, Journal of Experimental Social Psychology
Despite the modern rarity with which people are visual witness to moral transgressions involving physical harm, such transgressions are more accessible than ever thanks to their availability on social media and in the news. On one hand, the literature suggests that people form fast moral impressions once they already know what has transpired (i.e., who did what to whom, and whether there was harm involved). On the other hand, almost all research on the psychological bases for moral judgment has used verbal vignettes, leaving open the question of how people form moral impressions about observed visual events. Using a naturalistic but well-controlled image set depicting social interactions, we find that observers are capable of ‘moral thin-slicing’: they reliably identify moral transgressions from visual scenes presented in the blink of an eye (< 100 ms), in ways that are surprisingly consistent with judgments made under no viewing-time constraints. Across four studies, we show that this remarkable ability arises because observers independently and rapidly extract the ‘atoms’ of moral judgment (i.e., event roles, and the level of harm involved). Our work supports recent proposals that many moral judgments are fast and intuitive and opens up exciting new avenues for understanding how people form moral judgments from visual observation.
Innate sensitivity to face-to-face biological motion
2024, iScience
Sensitivity to face-to-face stimuli configurations, which likely indicates interaction, seems to appear early in infants’ development, and recently a preference for face-to-face (vs. other spatial configurations) has been shown to occur in macaque monkeys. It is unknown, however, whether such a preference is acquired through experience or as an evolutionary-given biological predisposition. Here, we exploited a precocial social animal, the domestic chick, as a model system to address this question. Visually naive chicks were tested for their spontaneous preferences for face-to-face vs. back-to-back hen dyads of point-light displays depicting biological motion. We found that female chicks have a spontaneous preference for the facing interactive configuration. Males showed no preference, as expected due to the well-known low social motivation of males in this highly polygynous species. These findings support the idea of an innate and sex-dependent predisposition toward social and interacting stimuli in a vertebrate brain such as that of chicks.
Not socially blind: Unimpaired perception of social interaction in schizophrenia
2024, Schizophrenia Research
Converging evidence that left extrastriate body area supports visual sensitivity to social interactions
2024, Current Biology
Navigating our complex social world requires processing the interactions we observe. Recent psychophysical and neuroimaging studies provide parallel evidence that the human visual system may be attuned to efficiently perceive dyadic interactions. This work implies, but has not yet demonstrated, that activity in body-selective cortical regions causally supports efficient visual perception of interactions. We adopt a multi-method approach to close this important gap. First, using a large fMRI dataset (n = 92), we found that the left hemisphere extrastriate body area (EBA) responds more to face-to-face than non-facing dyads. Second, we replicated a behavioral marker of visual sensitivity to interactions: categorization of facing dyads is more impaired by inversion than non-facing dyads. Third, in a pre-registered experiment, we used fMRI-guided transcranial magnetic stimulation to show that online stimulation of the left EBA, but not a nearby control region, abolishes this selective inversion effect. Activity in left EBA, thus, causally supports the efficient perception of social interactions.
Hierarchical organization of social action features along the lateral visual pathway
2023, Current Biology
Recent theoretical work has argued that in addition to the classical ventral (what) and dorsal (where/how) visual streams, there is a third visual stream on the lateral surface of the brain specialized for processing social information. Like visual representations in the ventral and dorsal streams, representations in the lateral stream are thought to be hierarchically organized. However, no prior studies have comprehensively investigated the organization of naturalistic, social visual content in the lateral stream. To address this question, we curated a naturalistic stimulus set of 250 3-s videos of two people engaged in everyday actions. Each clip was richly annotated for its low-level visual features, mid-level scene and object properties, visual social primitives (including the distance between people and the extent to which they were facing), and high-level information about social interactions and affective content. Using a condition-rich fMRI experiment and a within-subject encoding model approach, we found that low-level visual features are represented in early visual cortex (EVC) and middle temporal (MT) area, mid-level visual social features in extrastriate body area (EBA) and lateral occipital complex (LOC), and high-level social interaction information along the superior temporal sulcus (STS). Communicative interactions, in particular, explained unique variance in regions of the STS after accounting for variance explained by all other labeled features. Taken together, these results provide support for representation of increasingly abstract social visual content—consistent with hierarchical organization—along the lateral visual stream and suggest that recognizing communicative actions may be a key computational goal of the lateral visual pathway.
Seeing social interactions
2023, Trends in Cognitive Sciences
Seeing the interactions between other people is a critical part of our everyday visual experience, but recognizing the social interactions of others is often considered outside the scope of vision and grouped with higher-level social cognition like theory of mind. Recent work, however, has revealed that recognition of social interactions is efficient and automatic, is well modeled by bottom-up computational algorithms, and occurs in visually-selective regions of the brain. We review recent evidence from these three methodologies (behavioral, computational, and neural) that converge to suggest the core of social interaction perception is visual. We propose a computational framework for how this process is carried out in the brain and offer directions for future interdisciplinary investigations of social perception.

View all citing articles on Scopus

View full text

ViewpointTwos in human visual perception

Abstract

Section snippets

Sociality shapes perception

Visual tuning for facing dyads

Attentional advantage of facing dyads

Neural tuning and representational sharpening

Grouping of bodies: effects of access versus encoding

Multiple-body versus multiple-object perception

Conclusions

Acknowledgments

Neuroimage

Cognition

Cognition

Vision Research

Trends Cogn Sci

Trends Cogn Sci

Cortex; a Journal Devoted To the Study of the Nervous System and Behavior

Neuroimage

Cognition

Cognition

Neuropsychologia

Neuroimage

The representation of two-body shapes in the human visual cortex

The Journal of Neuroscience: the Official Journal of the Society for Neuroscience

How to build a baby that can read minds: Cognitive mechanisms in mindreading

The maladapted mind: Classic readings in evolutionary psychopathology

Human social attention

Annals of the New York Academy of Sciences

Interaction between scene and object processing revealed by human fMRI and MEG decoding

The Journal of Neuroscience: the Official Journal of the Society for Neuroscience

Configural face processing: A meta-analytic survey

Perception

Becoming a face expert

Philosophical Transactions of the Royal Society of London B Biological Scienceshilos Trans R Soc Lond B Biol Sci

Concept innateness, concept continuity, and bootstrapping

The Behavioral and Brain Sciences

Visual search for object categories is predicted by the representational architecture of high-level visual cortex

Journal of Neurophysiology

Featural and configurational processes in the recognition of faces of different familiarity

Perception

Two equals one: two human actions during social interaction are grouped as one unit in working memory

Psychological Science

Cognition does not affect perception: Evaluating the evidence for “top-down” effects

The Behavioral and Brain Sciences

Getting the gist of events: Recognition of two-participant actions from brief displays

Journal of Experimental Psychology. General

Viewpoint
Twos in human visual perception