Rhythmic parsing

1 Introduction

Readers generate from the graphemic string an intrinsic auditory version of the text entailing rich prosodic structure. Various reading studies have revealed that the silent prosodic rendition, called “implicit prosody,” may affect the syntactic analysis of written text. Several studies on “implicit prosody” suggest that readers’ preferences concerning the prosodic representation compete with preferences with respect to the syntactic analysis. Prosodic preferences may be particularly forceful when there are no strong syntactic preferences. As an example, Fodor (1998) argues that the preference for roughly equal-sized prosodic phrases affects readers’ attachment decisions in ambiguous environments like (1). In order to achieve a balanced output, a prosodic phrase boundary would separate teacher’s and mother in (1-a) while, in (1-b), the genitive NP and the head noun are phrased together. Via syntax-prosody mapping constraints, this difference in prosodic phrasing impinges on the syntactic interpretation, with the adjective attaching to the genitive NP in (1-a) but attaching to the NP_Gen-NP complex in (1-b).

However, the role of implicit prosody has been described as paradoxical (Fodor, 2002): on the one hand, it is assumed that prosody is shaped according to the syntactic structure assigned to the word string, suggesting that the syntactic analysis predetermines much of the prosodic representation; on the other hand, experimental evidence attests a clear influence of “implicit prosody” on the syntactic analysis proper. Research on implicit prosody has therefore concentrated on the question concerning at what stage of the syntactic analysis phonological factors constrain the parse. Clearly, the balance principle responsible for the attachment preferences in (1) may only apply once all relevant words are in the parser’s processing window. Correspondingly, Bader (1998) suggests a late influence of the prosodification on the syntactic analysis in reading. In his Prosodic Constraint on Reanalysis, it is proposed that prosodic factors add to the burden of syntactic reanalysis during sentence processing when the revised syntactic structure necessitates prosodic adjustments, too. Similarly, Augurzky (2006) claims that readers leave the prosodic rendition of the sentence underspecified during the initial processing stage, relying on purely syntactic cues. Only later are the syntactic parse and the prosodic rendition integrated. Hirose (2003) and Hwang and Steinhauer (2011) suggest that already during first pass parsing, syntactic analysis and prosodic representation are integrated, advocating early interaction of these domains in processing. Their experiments concern prosodic balance with respect to syntactic attachment preferences for long versus short phrases. It has to be noted, though, that the evaluation of phrase length by the parser requires the syntactic formation of these phrases in the first place. In this respect, the prosodification is dependent on at least limited syntactic pre-processing in these studies.

In the following section (Section 2), we review experimental evidence that challenges the idea that readers build prosodic structure only on the basis of syntactic pre-processing. Conversely, the experiments suggest that, at points of syntactic underspecification, phonological constraints alone may guide syntactic structure assignment in reading. Beyond its psycholinguistic importance, such evidence has repercussions for the architecture of the competence grammar that the parser consults during processing. We argue that the grammar has to be devised in such a way as to allow phonological influence on syntactic structure assignment. On the basis of the empirical evidence, we propose and advocate a parsing model which makes explicit reference to an optimality theoretic competence grammar integrating constraints from the domains of syntax, phonology and the corresponding interface (Section 3). We compare this competence-based model to other parsing models in Section 4. The paper ends with a conclusion in Section 5.

2 Experiment

Germanic languages have a general preference for the alternation of strong and weak syllables (e. g. Hayes, 1995; Liberman and Prince, 1977; Selkirk, 1984). It has been shown that a clash of two stressed syllables is avoided whenever more rhythmic alternatives are available. Faced with a potential stress clash, speakers might resort to stress shift (Bohn et al., 2011; Kiparsky, 1966; Visch, 1999) or they might choose a word order that prevents stress clash in the first place (Anttila et al., 2010; Ingason, 2015; Schlüter, 2005; Shih et al., 2015). If, in silent reading, readers indeed generate a speech-like prosodic representation of the text, as proposed in Ashby and Martin (2008), clash avoidance should also hold in the written modality. That is, stress clash avoidance should have consequences for the syntactic processing of the written string: in the face of an ambiguous structure that involves a stress clash in one reading but not in the other, there should be a preference for the version without stress clash. This hypothesis has recently been confirmed in two reading experiments (Kentner, 2012), which we summarize briefly. ^[1]

The object of investigation was temporarily ambiguous structures, with two possible interpretations of the word mehr and specific prosodic representations for each of the syntactic interpretations (2). The word mehr is either the unaccented part of the temporal adverbial nicht mehr (Engl.: “no longer”) (temp) or the obligatorily accented, comparative quantifier (Engl.: “more”) (comp). In order to test the rhythmic influence on syntactic parsing, the prosodically ambiguous word mehr was followed by a tri-syllabic verb with either initial (ini) or medial (med) stress, yielding four experimental conditions. The sentences are disambiguated in the phrase that ends the sentence. In the temp-conditions, the disambiguating material is an extraposed sentential complement of the verb, introduced by a complementizer or wh-pronoun. The comp-conditions are invariably disambiguated by the als-phrase, serving as the standard of comparison (i. e. the argument to the comparative mehr).

In the case of the temporal adverbial nicht mehr (temp), the two graphemic words form a single lexical item since the meaning cannot be decomposed any further. Lexical stress (marked by single underlines) falls on nicht, and mehr remains unstressed. Correspondingly, the phrasal accent (marked by small caps) assigned to the adverb falls onto the stressed nicht. In adverb–verb sequences, the verb bears the main or nuclear accent (marked by small caps and double underlines) (Truckenbrodt, 2006). As for the comp-conditions, mehr receives main phrase accent as it serves as a (comparative) complement to the verb; the verb itself may remain unaccented (Truckenbrodt, 2006).

Comparative, and thus accented, mehr followed by initial stress on the verb engenders a stress clash (comp-ini). It was hypothesized that, without disambiguating information, readers should initially favor the unaccented, temporal interpretation of mehr in order to avoid the stress clash. Hence, in the clash-condition, readers should be forced to reanalyze when encountering the disambiguating region (the phrase at the end of the sentence) and thus experience increased processing demand.

First, an oral reading experiment (unprepared reading) was set up with 24 sets of sentences like (2). Twenty-four participants were asked to read single sentences from a screen without preparation. In order to prevent look-ahead to the disambiguating material at the end of the sentence, readers were asked to start reading out loud as soon as the sentence was displayed on screen. The accentuation patterns were evaluated by two student assistants, who were presented the sound files up to the verb. Their judgments confirmed the hypothesis that readers, as long as they were unaware of the disambiguating information, avoided accentuation of mehr when it was followed by a verb with initial stress. Specifically, readers accented 28 % of mehr that were followed by a verb with medial stress but significantly fewer cases of mehr (19 %) when a verb with initial stress followed. These numbers also reveal a strong general preference for the unaccented version of mehr.

Further analysis demonstrated that readers significantly slowed down in speech at the disambiguating region (i. e. in the phrase after the verb complex) when their accentuation of mehr turned out to be inappropriate relative to the disambiguating information. This slowdown is indicative of a garden path effect, suggesting that the readers’ decision for accentuation of mehr involved a syntactic commitment to the relevant reading.

In a second experiment, 48 participants read the same set of sentences silently on screen with an eye-tracking device monitoring the fixation patterns. The evaluation of the eye-movement record in the disambiguating region attests significantly higher reading costs for the comp- conditions. Arguably, the difficulties associated with the comp-conditions reflect the general preference for the unaccented, temporal version of mehr that was observed in the oral reading experiment. On top of this main effect, the experiment yielded a significant interaction: reading times were significantly increased in the stress clash condition comp-ini as compared to the other, rhythmically innocuous conditions (see Figure 1).

Figure 1:

Re-reading probability, re-reading time and total fixation time within the disambiguating region broken down by condition. The highest reading times and fixation probabilities were obtained for the stress clash condition comp-ini.

The results are interpreted with recourse to the preference for rhythmic alternation, which apparently prevails even in silent reading. That is, the general preference for the unaccented, temporal-adverbial reading is reinforced by the rhythmic environment in the comp-ini condition. In order to avoid a potential stress clash, readers avoid an “implicit” accent on mehr and, correspondingly, follow the generally preferred temporal-adverbial reading. The temporal-adverbial reading, however, is incompatible with the disambiguating information, which causes the observed reading difficulties in the comp-ini condition. Since, in the comp-med condition, there is no danger of stress clash, the preference for the temporal-adverbial reading is significantly weaker. The results support the claim that the direct rhythmic environment, i. e. the lexical stress on the verb, affects the earliest stage of syntactic processing, viz. the determination of the syntactic category of the preceding ambiguous item mehr.

3 Integrating comprehension and production in an Optimal Parsing account

Optimality Theory (OT) as a theory of grammar was originally designed to explain the relation between assumed underlying structures (input) and surface structures (output) in phonology. Input and output can be construed in various ways, however, depending on the nature of the task the grammar is used for.

Crucially, the same grammar can be applied to all kinds of linguistic performance. The grammar is independent of input and output modality but it is central to both. For example, in OT accounts that deal with language comprehension, the input is understood as the phonetic signal the listener is exposed to and the output is the mental representation he builds or accesses on the basis of the signal (e. g. Smolensky, 1996; Beaver and Lee, 2004). Similarly, OT can be applied to reading. In reading, the written string of lexical items can be construed as input and the output might be a syntactic description assigned to the string or the phonological representation thereof, which could be used for oral reading.

The OT grammar is thus a device that interprets (in the case of syntactic structure assignment) and generates (in the case of phonologization) linguistic objects; it is parser and synthesizer at the same time.

The principle of constraint violability, which is central to OT, implies that the degree of adherence to the grammar can be evaluated for complete sentences as well as for partial, uncompleted, even ungrammatical candidate structures. This is a most desirable feature for any system of linguistic competence that aspires to have explanatory power for language performance. After all, being engaged in sentence comprehension or production means dealing with partial and imperfect linguistic objects at least as long as the sentence being processed is unfinished, arguably most of the time.

OT has the capability to build outputs for incremental input. It has been applied to sentence parsing and proven to have the capacity to model a wide range of sentence processing phenomena (Fanselow et al., 1999; Hoeks and Hendriks, 2011; Stevenson and Smolensky, 2006). The rationale is simple: scanning the input incrementally from left to right (or in chronological order, in the case of auditory language comprehension), for any given piece of input, the OT parser would favor the analysis that best satisfies the constraint hierarchy.

Processing difficulty arises when high ranking constraints become relevant with additional input and force the parser to drop the currently optimal candidate in favor of a previously suboptimal one. The idea that constraints might become relevant with new input in parsing implicates that the order in which information is assessed by the parser is determined by the input stream. Depending on the structure of the input, constraints from different linguistic domains may interact freely at any time during the parse. That is, the evaluation process is cross-modular if the output is cross-modular. For example, constraints regulating the well-formedness of prosodic structure may interact with syntactic requirements in determining the structure of a linguistic object. In this respect, OT crucially differs from models of sentence processing that assume a fixed order in which different kinds of information are considered by the language processor at any given step of the analysis. For example, it is not the case in cross-modular OT that semantic and contextual information is only consulted after a first syntactic analysis of the input has been completed (as has been suggested in some models of sentence processing, e. g. Garden Path Theory, Frazier 1987, Friederici 2002) and a great deal of psycholinguistic evidence supports the parallel engagement of contextual semantic and syntactic knowledge in sentence parsing (e. g. Garnsey et al., 1997; Trueswell et al., 1994).

For the present case, the simultaneous application of constraints of various linguistic domains (syntax and phonology), the capability of evaluating incremental input, and the possibility to integrate both parsing and synthesizing of linguistic objects within a single framework are well suited for modeling the interaction of prosodification and syntactic structure building in reading performance. The OT model used for this endeavor has the following characteristics:

1. The written string of lexical items serves as input.

2. The output consists of both a syntactic description of the input and its prosodic rendition.

3. The relevant constraints evaluate syntactic well-formedness, phonological well-formedness, and the agreement between syntactic and phonological structure.

The general method for testing whether OT can be used to model the parsing facts is as follows:

First, we determine the constraints relevant for the type of ambiguity under consideration. We make sure that the constraints are well motivated and have independent support. Second, we ascertain the ranking among the constraints as established in the relevant literature or on the basis of grammaticality judgments. Third, we verify the validity of the model by reference to the reading performance on the nicht mehr-ambiguity detailed in the previous section.

We assume that the model simultaneously generates two kinds of structured representations which are combined: a syntactic interpretation and a prosodic rendition. The written input provides information about the lexical items and their sequence. The combination of syntactic and prosodic structure is evaluated against the constraint hierarchy. The syntactic descriptions used here conform to standard assumptions concerning phrase structure, i. e. syntactic parses may generally be represented by a fully connected predicate argument structure with a single root representing the phrase or the sentence.

4 Discussion

The performance differences the model predicts for the four conditions at parsing step 3 do indeed appear to reflect the reading data obtained in the experiments. Little processing difficulty is predicted for both conditions with the adverbial reading (Tableau 3). This is obvious for the condition with the initially stressed verb. The optimal candidate is syntactically and prosodically consistent with the optimal candidates at both previous parsing steps (hence the three pointing fingers). In the case of the verb with medial stress, the deviating parsing preference that is predicted for the second parsing step is a very weak one. It hinges on a low ranking constraint and might thus be easily overwritten in the third step.

Turning to the two conditions featuring the comparative reading (Tableau 4), a clear difference in processing difficulty is predicted between cases with initial and medial stress on the verb. In the latter condition, a weak preference that had been established at the second parsing step is confirmed. The structure should therefore be processed relatively easily compared to the comparative mehr followed by a verb with initial stress. It is this condition that is predicted to produce the highest processing costs: the competitor that was the optimal candidate in the two previous parsing steps founders on a high ranking constraint.

This prediction of the model is borne out in the actual data of the reading experiment. In the disambiguating region, several dependent variables reveal an interaction between the factors “syntactic reading” and “verbal stress pattern” that is mainly due to the striking reading difficulty observed for the condition that forces a stress clash (or else a deviation from the lexically determined stress pattern), i. e. comparative mehr followed by initial stress on the verb.

The model is also compatible with the general prevalence of unaccented readings of mehr that was observed in the oral reading experiment (only 25 % of cases accented; cf. summary of oral reading experiment). The avoidance of accent on mehr can be explained with recourse to the model’s first parsing step (cf. Tableau 1), in which the adverbial reading was established as the optimal interpretation, and the competing comparative version ruled out by strong syntactic constraints. Since, in the oral reading experiment, readers were encouraged to read as fluently as possible without self-corrections, this initial analysis may have been only mildly affected by the incoming verb in the second parsing step. ^[10]

Generally, the proposed OT parser accounts for the prosodic effects that were shown to act upon syntactic ambiguity resolution in reading. The OT model makes direct use of the grammar in determining its parsing preferences. The assumption is that grammatical restrictions are applied to incremental structure building and that preferences at points of ambiguity reflect these grammatical requirements. The constraint ranking that derives the parsing preferences has a sound and independent grammatical motivation and, in the present case, it can do without additional extra-grammatical processing constraints. Moreover, not only the constraints but also the specific ranking employed for this performance theory hold for the ordinary competence grammar. Importantly, this model is cross-modular in that it integrates constraints from different modules of grammar, namely syntax, phonology and the corresponding interface.

With that said, we shall briefly return to the problem of the syntax-phonology interface in grammar touched upon in the introduction. The present model is a performance-compatible grammatical device that allows phonological constraints to act upon syntactic structure building. This is especially evident in parsing step 2 (cf. Tableau 2, lower panel), in which the optimal candidate, complete with a syntactic description, is selected as optimal because it fares better than the relevant syntactic competitor only with respect to phonological constraints. Given that it is the OT grammar proper that produces this result, this parsing step constitutes an offense against the notion of phonology-free syntax as forcefully advocated by Pullum and Zwicky (1988). If this incremental model and its constraint hierarchy is realistic, the idea of a merely unidirectional relation between syntax and phonology is once more invalidated (see Inkelas and Zec, 1995; Rice and Svenonius, 1998; Schlüter, 2003; Zec and Inkelas, 1990, for more evidence against phonology-immune constituent structure building). Rather, this phenomenon is reminiscent of the idea that Bierwisch (1966) put forward referring to the role of prosody in language production:

Es ist sehr wohl möglich, dass ein Sprecher sich durch ein begonnenes Intonationsmuster zur Wahl einer bestimmten syntaktischen Struktur veranlasst fühlt […].

It is very well possible that a speaker feels himself prompted to choose a certain syntactic structure owing to the intonation pattern that he has started.

Bierwisch (1966), 105.; translation by the author

A grammar and/or processing model that does justice to phonological interference of this type must surely allow for at least limited interaction of syntax and prosody. Without further justification, I submit that situations of the kind found here (and formulated by Bierwisch, 1966) are not uncommon in incremental sentence processing. For the assignment of a syntactic description to a full sentence, however, recourse to phonology might be the exception rather than the rule.

In the following section I will compare the OT approach to other models of sentence processing.

5 Conclusion

The OT account of sentence reading outlined here has several advantages over other theories: First, the OT approach makes direct use of grammatical principles to determine parsing preferences at any given parsing step. The constraints and the ranking used to model the performance data have independent support from the general competence grammar. For the kind of ambiguity studied here, the OT grammar suffices to make clear and testable processing predictions. The present model achieves this goal without making reference to extra-grammatical features of the input (such as word frequency) or working memory constraints. Nevertheless, it might be possible to formulate extra-grammatical requirements in optimality theoretic terms that could be integrated into this framework to explain processing facts that are not reducible to core grammar. The architecture of the model is parsimonious and establishes clarity: a very small set of grammatical constraints allows the model’s predictions to be derived. Still, the present OT approach allows complex, non-trivial interactions of various grammatical modules in that it integrates constraints from syntax, phonology and the respective interface. That way, it allows the modeling of prosodic effects on syntactic ambiguity resolution that cannot easily be modeled in frameworks that assume a merely unidirectional syntax-phonology interface. Moreover, the model is easily scalable; it can be flexibly adapted and enlarged to fit phenomena in which semantic or contextual constraints interact with syntactic and phonological ones. What sets it apart from other constraint-based models is its modality independence. None of the constraints used here is specifically designed for the task the model was tested on, viz. reading. The same constraint hierarchy can potentially be used to make predictions about performance in listening, speaking or writing. That is, the model answers the call for a more integrative account of language production and comprehension – a demand that has been voiced again and again in psycholinguistics (e. g. Cutler and Norris, 1999; Ferreira, 2003; Garrett, 2000; Levelt et al., 1999; Pickering and Garrod, 2007).

Certainly, fundamental issues remain unresolved: For one thing, as discussed above, it is not trivial to relate the merely ordinal expression of processing preferences derived from the OT model to numerical data from psycholinguistic experiments. In addition, it is undisputed that factors like word frequency or the limitations of working memory are important forces in sentence comprehension and production. It would be far-fetched to assume that these factors are reducible to the workings of purely grammatical constraints. At this stage, it is unclear how frequency or memory effects could be integrated in the present OT model. In spite of these desiderata, the success of this model and its predecessors suggests that the general approach of applying OT to language performance is a fruitful one. The model introduced here can be seen as an extension of existing OT parsing accounts (Fanselow et al., 1999; Hoeks and Hendriks, 2011; Stevenson and Smolensky, 2006). The model shows how cross-modular effects in sentence processing and effects of production-driven comprehension may be captured within a unified model of linguistic competence and performance.