Research ReportThe contribution of phonological information to visual word recognition: Evidence from Chinese phonetic radicals
Introduction
There is a large agreement about the cortical circuits involved in word reading (see Fig. 1 for an overview, based on Dehaene, 2009). Like all visual input, written words first activate the primary visual cortex at the back of the brain. The incoming signal is further processed in higher-level areas of the visual cortex and proceeds anteriorly towards the temporal cortex, where one region responds particularly strongly to written verbal input. This ventral occipitotemporal (vOT) region has been called the Visual Word Form Area (VWFA; Cohen et al., 2002) and supports strong interactive connections with the brain areas involved in spoken word recognition and speech production, situated in temporal, parietal and frontal cortex (Fig. 1; see also Cai et al., 2008; Cai et al., 2010; Van der Haegen et al., 2012).
Two main routes connect the VWFA to the language-related brain areas (Carreiras et al., 2014; Taylor et al., 2013; Wandell & Le, 2017). The first route proceeds ventrally into the middle and anterior temporal cortex and subsequently, the inferior frontal cortex. This “ventral stream” is mainly involved in the activation of word meaning and whole-word phonology (the spoken word representation). The second route proceeds dorsally and connects the VWFA with the angular gyrus and the supramarginal gyrus in the parietal cortex, after which it also reaches the inferior frontal cortex. This route is called the dorsal route and is thought to predominantly mediate the sub-lexical interaction between written letters and spoken sounds. Because all connections are bidirectional, the two routes are thought to interact closely and to underlie visual word recognition together. Price and Devlin (2011) argued that the VWFA integrates the bottom-up visual input with top-down predictions generated from prior experience. As a source of top-down feedback, in particular the left inferior frontal gyrus (IFG) has been shown to modulate activity in the visual word form area in the early stages of visual word recognition (Woodhead et al., 2014).
In addition to the cortical areas traditionally investigated in visual word recognition research (summarized in Fig. 1), there is evidence for the involvement of the cerebellum (Alvarez & Fiez, 2018), subcortical structures ((Cocquyt et al., 2019)), and the insula (Borowsky et al., 2006) in word reading. For instance, Alvarez and Fiez (2018) reported the connectivity of the cerebellum to both the dorsal and the ventral pathway. They argued that this is because the cerebellum supports the development of fluent visual word recognition. Given that these regions were not a focus of our research, we will not discuss them further, although it may be good to keep them in mind for future research.
The network shown in Fig. 1 could be present in the left as well as in the right hemisphere of the brain. However, in the vast majority of people, the reading network is largely confined to the left hemisphere. Cai et al. (2008) argued that this is because speech production requires a single control center in the inferior frontal cortex. The many interactions between this region and the other language-related regions do not allow for multiple cross-hemispheric information exchanges because of the time costs involved (see also Brysbaert, 2004). As a result, the reading network largely lateralizes to the speech controlling hemisphere.
Lateralization requires the transfer of information from the right hemisphere to the left hemisphere in visual word recognition, as the word part to the left of the fixation position is initially sent to the occipital cortex in the right hemisphere (Ellis & Brysbaert, 2010; Van der Haegen et al., 2013). There are indications that part of the transfer already occurs before the information reaches the visual word form area (Barca et al., 2011; Chu & Meltzer, 2019; Selpien et al., 2015; Strother et al., 2016, 2017). In particular, Strother and colleagues presented evidence for an occipital word form area, in which considerably more transfer occurs from the right hemisphere to the left hemisphere than vice versa. The occipital word form area is situated in the inferior occipital cortex (Talairach coordinates x = −39, y = −80, z = −10), posterior to the visual word form area (Talairach coordinates x = −40, y = −59, z = −10). A similar distinction between the middle and posterior occipitotemporal sulcus was made by Lerma-Usabiaga et al. (2018), although their posterior part was slightly more anterior (y = −71.5). Stevens et al. (2017) presented evidence that the occipital word form area may already have functional connectivity with the other language areas shown in Fig. 1. Also in patients with pure alexia, there is evidence that the connections between the left occipital cortex and left vOT and IFG can be strengthened by reading training (Kerry et al., 2019; Woodhead et al., 2013).
The findings discussed so far were based on alphabetic languages (mainly English), in which words are written as sequences of letters referring to the sounds of the spoken word. This is different from the logographic Chinese script, where graphic forms (characters) map onto meanings (morphemes) rather than sounds. For instance, the written word给 (give) is pronounced as GEI in Mandarin, but as KAP in Cantonese. In addition, Chinese characters are formed with strokes in a square rather than in the linear structure of alphabetic words, and the visual complexity depends on the number of strokes in the character.
Because there are few sub-word relations between written symbols and pronunciation in Chinese (for an exception, see below), it has been suggested that the dorsal route is used differently in logographic word recognition than in alphabetic word recognition. For instance, (Kawabata Duncan et al., 2014) argued that the dorsal route contributes less to the recognition of logographic Kanji words in Japanese than to the recognition of alphabetic Hiragana words. Alternatively, Tan et al. (2005) argued that the dorsal route for Chinese word reading is based on whole-word phonology (which requires the written word to be recognized before the phonology can be activated; so-called addressed phonology). This involves the middle frontal gyrus together with a phonological short-term memory store in the inferior parietal cortex to keep the representation activated for a short period of time (Fig. 2). In contrast, alphabetic languages would make more use of letter-sound correspondences (assembled phonology), which involve the inferior frontal gyrus and a region consisting of the supramarginal gyrus in the ventral part of the inferior parietal cortex and the upper back part of the temporal gyrus. This conclusion, however, was questioned by R. Zhao, et al. (2017), who argued that the difference depends more on stimulus/task differences than language differences.
Another difference that has been postulated between logographic and alphabetic languages is a larger involvement of the right hemisphere. For instance, it has been argued that the vOT region in the right hemisphere is more active in logographic scripts than in alphabetic scripts. Hirshorn et al. (2016) reported experimental evidence for this difference. They taught participants a new language either with an alphabetic script or with a logographic script. Greater bilateral activation was observed in the visual word form area (Talairach coordinates ±45, −57, −12) for the logographic script than for the alphabetic script. More evidence for bilateral processing in Chinese word recognition was reported in meta-analyses (Bolger et al., 2005; Tan et al., 2005; Wu et al., 2012; Zhu et al., 2014).
Evidence for bilateral processing in Chinese word recognition was also presented by (Zhao et al., 2017a). By using words that differed in orthographic and phonological similarity, they showed that the effects of phonological similarity could be observed in the VWFA and in an anterior region of the fusiform gyrus. Importantly, in this study, the activation was very similar in the left and right cerebral hemispheres. This is different from the conclusion of a meta-analysis of R. Zhao et al. (2017) who reported clear left hemisphere dominance for the visual word form area in Chinese word processing. Given that speech production in the vast majority of Chinese people is left-lateralized (Mariën et al., 2004; Wang, 1996), it is reasonable to expect left dominance for Chinese visual word recognition as well, based on Cai et al. ’s (2008) argument that cross–hemisphere interactions hinder performance.
In our discussion so far we have considered the logographic and alphabetic scripts as two clearly separated categories. This does not correspond completely to reality. In alphabetic languages, the correspondence between the written letters (orthography) and word pronunciation (phonology) can be more or less transparent. Everybody with some knowledge of English pronounces the word “kiss” correctly. However, few people pronounce the word “awry” correctly without having learned the pronunciation explicitly. In irregular words such as “awry”, assembled (letter-to-sound) phonology is of little use and readers have to rely on addressed phonology.
Similarly, in a logographic script like Chinese, it is not true that written characters never include sub-word cues about the pronunciation. Over 80% of the 7000 most frequent characters contain phonetic radicals helping with the pronunciation (Li & Kang, 1993). The dominant type of Chinese characters consists of phonetic compounds, which have a phonetic radical and a semantic radical conveying information about character meaning.
A further interesting aspect of Chinese orthography is that the phonetic radical can be situated in the left or the right half of the character. According to an analysis by Hsiao and Shillcock (2006), more than 70% of phonetic compounds have a left-right configuration. Most of these have the phonetic radical to the right and the semantic radical to the left (sP characters); the remaining have the opposite arrangement (Ps characters). The ratio of sP characters to Ps characters is about 9 to 1.
The lateral position of the phonetic radical is interesting because it increases our chances of picking up the brain signal. Given that a phonetic radical in the left half of a Chinese character is sent to the right occipital cortex and a phonetic radical in the right half of a Chinese character to the left occipital cortex (Ellis & Brysbaert, 2010; Van der Haegen et al., 2013), chances are high that we will be able to pick up the brain signal involved in the processing of the phonetic radical by comparing activity in the left and the right brain half. Hsiao and colleagues (Hsiao et al., 2007; Hsiao & Liu, 2010) presented some evidence for this. In an EEG study, they reported that Chinese characters with the phonetic radical to the right elicited left-lateralized processing in an early ERP component (N1, around 180–200 ms post-stimulus), whereas no lateralized processing was observed for characters with the phonetic radical to the left. Source analysis suggested that the difference in N1 was located close to vOT.
More specifically, we may expect to see different brain activity due to the phonetic radical position in vOT, inferior parietal cortex, supramarginal gyrus, and/or in inferior/middle frontal cortex (Fig. 2).
To make sure that the effects are due to phonology and not to orthographic differences between the Chinese characters with phonetic radicals to the left or to the right, we made use of an additional set of Chinese character pairs that also have a left-right configuration but do not include phonetic radicals providing a clue about the pronunciation of the word. Such words are referred to as ideographs. For example, the ideograph character ‘休’ (xiū, meaning rest), shows a person (人, rén) resting by a tree (木, mù) and does not include a phonetic radical. Such words provide an opportunity to control for the orthographic information in the radicals, as they have the same left-right structure but do not have sub-word clues about their pronunciation.
Section snippets
Participants
Forty healthy graduate students (17 males; mean age: 26.85 years; range: 23–34 years) were paid €25 to take part in the experiment. All participants were native literate Mandarin speakers with normal or corrected-to-normal vision, were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971), and had no history of neurologic or psychiatric problems. All participants were studying at Ghent University and screened to meet all conditions for scanning. Before the experiment,
Behavioral results
No participant reported any difficulty in completing the color word searching task and participants made on average .6% errors. Importantly, the number of button presses in the critical blocks was less than once per participant. The few blocks with button presses were omitted from the fMRI analyses.
fMRI whole-brain analysis
First, we ran a whole-brain analysis in which we compared the words with critical information in the right half to the words with critical information in the left half for each type of word (Table 2
Discussion
In this study, we sought to obtain more information about the role of phonology in visual word recognition by examining brain activity in response to Chinese characters containing sub-lexical phonetic information. In particular, we compared characters with a phonetic radical in the left half to characters with a phonetic radical in the right half (respectively Ps and sP characters). These characters have been used before by Hsiao and colleagues to study Chinese word processing (Hsiao et al.,
CRediT authorship contribution statement
Xiaodong Liu: Investigation, Formal analysis, Data Curation, Writing - original draft, Writing - review & editing. Luc Vermeylen: Investigation, Writing - Original Draft. David Wisniewski: Formal analysis, Writing - Original Draft. Marc Brysbaert: Conceptualization, Methodology, Writing - original draft; Writing - review & editing, Supervision, Funding acquisition.
Open Practices
The study in this article earned Open Materials and Open Data badges for transparent practices. Materials and data for the study are available at https://osf.io/5k2wn/. We report how we determined our sample size, all data exclusions (if any), all inclusion/exclusion criteria, whether inclusion/exclusion criteria were established prior to data analysis, all manipulations, and all measures in the study. No part of the study procedures and analyses were pre-registered prior to the research being
Declaration of competing interest
The authors declare that they have no financial disclosures or potential conflicts of interests.
Acknowledgements
We would like to thank Qing Cai for her help with the stimulus selection and Helena Verhelst for help with the collection of data. This work was supported by a grant awarded by the Ghent University (CCN-kas D/01329/01, Belgium) to prof. Marc Brysbaert and a studentship to Xiaodong Liu (Oversea Study Program of Guangzhou Elite Project, JY201717, China).
References (67)
- et al.
Current perspectives on the cerebellum and reading development
Neuroscience and Biobehavioral Reviews
(2018) - et al.
Language lateralization and the role of the fusiform gyrus in semantic processing in young children
Neuroimage
(2006) - et al.
The neural basis of the right visual field advantage in reading: An MEG analysis using virtual electrodes
Brain and Language
(2011) The importance of interhemispheric transfer for foveal vision: A factor that has been overlooked in theories of visual word recognition and object perception
Brain and Language
(2004)- et al.
The what, when, where, and how of visual word recognition
Trends in Cognitive Sciences
(2014) - et al.
The involvement of subcortical grey matter in verbal semantic comprehension: A systematic review and meta-analysis of fMRI and PET studies
Journal of Neurolinguistics
(2019) AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages
Computers and Biomedical Research
(1996)- et al.
Split fovea theory and the role of the two cerebral hemispheres in reading: A review of the evidence
Neuropsychologia
(2010) - et al.
Foveal splitting causes differential processing of Chinese orthography in the male and female brain
Cognitive Brain Research
(2005) - et al.
Neural correlates of foveal splitting in reading: Evidence from an ERP study of Chinese character recognition
Neuropsychologia
(2007)
Recognition processes in character naming
Evaluation of the dual route theory of reading: A metanalysis of 35 neuroimaging studies
Neuroimage
Reliable individual-level neural markers of high-level language processing: A necessary precursor for relating neural variability to behavioral and genetic variability
Neuroimage
Adult crossed aphasia in dextrals revisited
Cortex; a Journal Devoted To the Study of the Nervous System and Behavior
The assessment and analysis of handedness: The Edinburgh inventory
Neuropsychologia
Visual word recognition: The first half second
Neuroimage
The Interactive Account of ventral occipitotemporal contributions to reading
Trends in Cognitive Sciences
Left dominance for language perception starts in the extrastriate cortex: An ERP and sLORETA study
Behavioural Brain Research
An fMRI study of visual hemifield integration and cerebral lateralization
Neuropsychologia
Top-down modulation of ventral occipito-temporal responses during visual word recognition
Neuroimage
Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain
Neuroimage
Colateralization of Broca's area and the visual word form area in left-handers: fMRI evidence
Brain and Language
Phenotypes in hemispheric functional segregation? Perspectives and challenges
Physics of Life Reviews
Diagnosing the neural circuitry of reading
Neuron
Automatic semantic influence on early visual word recognition in the ventral occipito-temporal cortex
Neuropsychologia
LI-tool: A new toolbox to assess lateralization in functional MR-data
Journal of Neuroscience Methods
A combined bootstrap/histogram analysis approach for computing a lateralization index from neuroimaging data
Neuroimage
A meta-analysis of fMRI studies on Chinese orthographic, phonological, and semantic processing
Neuroimage
Rethinking the function of brain regions for reading Chinese characters in a meta-analysis of fMRI studies
Journal of Neurolinguistics
Neural basis of phonological processing in second language reading: An fMRI study of Chinese regularity effect
Neuroimage
Parafoveal-on-foveal interactions in word recognition
The Quarterly Journal of Experimental Psychology Section A
Cross-cultural effect on the brain revisited: Universal structures plus writing system variation
Human Brain Mapping
FMRI of ventral and dorsal processing streams in basic reading processes: Insular sensitivity to phonology
Brain Topography
Cited by (8)
Cross-Modal Impact of Recent Word Encountering Experience
2024, Scientific Studies of ReadingBuilding the VisSE Corpus of Spanish SignWriting
2023, Language Resources and EvaluationNeural representation of phonological information during Chinese character reading
2022, Human Brain MappingThe Representations of Chinese Characters: Evidence from Sublexical Components
2022, Journal of Neuroscience