Skip to main content
SpringerLink
Log in

Not all arguments are processed equally: a distributional model of argument complexity

  • Original Paper
  • Published:
Language Resources and Evaluation Aims and scope Submit manuscript

Abstract

This work addresses some questions about language processing: what does it mean that natural language sentences are semantically complex? What semantic features can determine different degrees of difficulty for human comprehenders? Our goal is to introduce a framework for argument semantic complexity, in which the processing difficulty depends on the typicality of the arguments in the sentence, that is, their degree of compatibility with the selectional constraints of the predicate. We postulate that complexity depends on the difficulty of building a semantic representation of the event or the situation conveyed by a sentence. This representation can be either retrieved directly from the semantic memory or built dynamically by solving the constraints included in the stored representations. To support this postulation, we built a Distributional Semantic Model to compute a compositional cost function for the sentence unification process. Our evaluation on psycholinguistic datasets reveals that the model is able to account for semantic phenomena such as the context-sensitive update of argument expectations and the processing of logical metonymies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. We represent dependencies according to the Universal Dependencies annotation scheme: http://universaldependencies.org/.

  2. Beyond traditional calculations of thematic fit for the fillers of verb roles, we also compute scores for a generic co-participant relation between filler nouns, as experimental studies report processing facilitations also due to inter-arguments typicality (e.g. the facilitation for sentences with typical agent–patient combinations in Bicknell et al. 2010).

  3. Although the architecture presented here is similar to the proposals in Chersoni et al. (2016a) and Chersoni et al. (2017a), several details of the framework have been changed, and thus the described results are different.

  4. Corpora were preprocessed with the pos-tagger described in Dell’Orletta (2009) and the dependency parser by Attardi et al. (2009).

  5. The scores of the baselines are not reversed as the ArgComp ones and they are comparable to the thematic fit scores of the \(\theta \) component. Thus, the task for the baselines is to assign lower scores to the incongruent condition.

  6. The accuracy score has been provided by the author himself.

  7. p-values have been computed with the \(\tilde{\chi }^{2}\) test.

  8. Importantly, the covert events do not contribute to the \(\sigma \) scores, since the corresponding verbs are not present in the linguistic input.

  9. All p-values were computed with the \(\chi ^{2}\) test.

  10. Since the computation of the two \(\theta \)s in ThetaProd requires a different number n of factors, the scores have been normalized by elevating them to the power of 1/n.

References

  • Attardi, G., Dell’Orletta, F., Simi, M., & Turian, J. (2009). Accurate dependency parsing with a stacked multilayer perceptron. In Proceedings of EVALITA.

  • Baggio, G., & Hagoort, P. (2011). The balance between memory and unification in semantics: A dynamic account of the N400. Language and Cognitive Processes, 26(9), 1338–1367.

    Article  Google Scholar 

  • Baggio, G., Van Lambalgen, M., & Hagoort, P. (2012). The processing consequences of compositionality. In The Oxford handbook of compositionality (pp. 657–674). Oxford: Oxford University Press.

  • Baroni, M., & Lenci, A. (2010). Distributional memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4), 673–721.

    Article  Google Scholar 

  • Baroni, M., Bernardini, S., Ferraresi, A., & Zanchetta, E. (2009). The WaCky Wide Web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation, 43(3), 209–226.

    Article  Google Scholar 

  • Bicknell, K., Elman, J. L., Hare, M., McRae, K., & Kutas, M. (2010). Effects of event knowledge in processing verbal arguments. Journal of Memory and Language, 63(4), 489–505.

    Article  Google Scholar 

  • Blache, P. (2011). Evaluating language complexity in context: New parameters for a constraint-based model. In CSLP-11, workshop on constraint solving and language processing.

  • Blache, P. (2016). Representing syntax by means of properties: A formal framework for descriptive approaches. Journal of Language Modelling, 4(2), 183–224.

    Article  Google Scholar 

  • Chersoni, E., Blache, P., & Lenci, A. (2016a). Towards a distributional model of semantic complexity. In Proceedings of the COLING workshop on computational linguistics for linguistic complexity (CL4LC).

  • Chersoni, E., Santus, E., Lenci, A., Blache, P., & Huang, C. R. (2016b). Representing verbs with rich contexts: An evaluation on verb similarity. In Proceedings of EMNLP.

  • Chersoni, E., Lenci, A., & Blache, P. (2017a). Logical metonymy in a distributional model of sentence comprehension. In Proceedings of *SEM.

  • Chersoni, E., Santus, E., Blache, P., & Lenci, A. (2017b). Is structure necessary for modeling argument expectations in distributional semantics? In Proceedings of IWCS.

  • Chersoni, E., Santus, E., Pannitto, L., Lenci, A., Blache, P., & Huang, C. R. (2019). A structured distributional model of sentence meaning and processing. Natural Language Engineering, 25(4), 483–502.

    Article  Google Scholar 

  • Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12, 2493–2537.

    Google Scholar 

  • Dell’Orletta, F. (2009). Ensemble system for part-of-speech tagging. In Proceedings of EVALITA.

  • Delogu, F., Crocker, M. W., & Drenhaus, H. (2017). Teasing apart coercion and surprisal: Evidence from eye-movements and ERPs. Cognition, 161, 46–59.

    Article  Google Scholar 

  • Elman, J. L. (2009). On the meaning of words and dinosaur bones: Lexical knowledge without a lexicon. Cognitive Science, 33(4), 547–582.

    Article  Google Scholar 

  • Elman, J. L. (2014). Systematicity in the lexicon: On having your cake and eating it too. In P. Calvo & J. Symons (Eds.), The architecture of cognition: Rethinking Fodor and Pylyshyn’s systematicity challenge. Cambridge, MA: The MIT Press.

    Google Scholar 

  • Erk, K. (2007). A simple, similarity-based model for selectional preferences. In Proceedings of ACL.

  • Erk, K., Padó, S., & Padó, U. (2010). A flexible, corpus-driven model of regular and inverse selectional preferences. Computational Linguistics, 36(4), 723–763.

    Article  Google Scholar 

  • Evert, S. (2004). The statistics of word cooccurrences: Word pairs and collocations. PhD Thesis, University of Stuttgart.

  • Frisson, S., & McElree, B. (2008). Complement coercion is not modulated by competition: Evidence from eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(1), 1–11.

    Google Scholar 

  • Gibson, E. (2000). The dependency locality theory: A distance-based theory of linguistic complexity. In Image, Language, Brain (pp. 95–26). Cambridge, MA: MIT Press.

  • Greenberg, C., Sayeed, A. B., & Demberg, V. (2015). Improving unsupervised vector-space thematic fit evaluation via role-filler prototype clustering. In Proceedings of HLT-NAACL.

  • Hagoort, P. (2013). MUC (memory, unification, control) and beyond. Frontiers in Psychology, 4, 416.

    Google Scholar 

  • Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL-HLT.

  • Hale, J. (2016). information-theoretical complexity metrics. Language and Linguistics Compass, 10(9), 397–412.

    Article  Google Scholar 

  • Hare, M., Jones, M., Thomson, C., Kelly, S., & McRae, K. (2009). Activating event knowledge. Cognition, 111(2), 151–167.

    Article  Google Scholar 

  • Heinzerling, B., Moosavi, N. S., & Strube, M. (2017). Revisiting selectional preferences for coreference resolution. In Proceedings of EMNLP.

  • Jurafsky, D., & Martin, J. H. (2014). Speech and language processing (Vol. 3). London: Pearson.

    Google Scholar 

  • Kamp, H. (2013). Meaning and the dynamics of interpretation: Selected papers by Hans Kamp. Leiden: Brill.

    Book  Google Scholar 

  • Kuperberg, G. R. (2016). Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events. Language, Cognition and Neuroscience, 31(5), 602–616.

    Article  Google Scholar 

  • Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207(4427), 203–205.

    Article  Google Scholar 

  • Lapata, M., & Lascarides, A. (2003). A probabilistic account of logical metonymy. Computational Linguistics, 29(2), 261–315.

    Article  Google Scholar 

  • Lascarides, A., & Copestake, A. (1998). Pragmatics and word meaning. Journal of Linguistics, 34, 378–414.

    Article  Google Scholar 

  • Lauwers, P., & Willems, D. (2011). Coercion: Definition and challenges, current approaches, and new trends. Linguistics, 49(6), 1219–1235.

    Article  Google Scholar 

  • Lebani, G. E., & Lenci, A. (2018). A distributional model of verb-specific semantic roles inferences. In Language, cognition, and computational models (pp. 118–158). Cambridge: Cambridge University Press.

  • Leech, G. N. (1992). 100 Million words of English: The British National Corpus (BNC). In Second language research.

  • Lenci, A. (2011). Composing and updating verb argument expectations: A distributional semantic model. In Proceedings of the ACL workshop on cognitive modeling and computational linguistics.

  • Lenci, A. (2018). Distributional models of word meaning. Annual Review of Linguistics, 4, 151–171.

    Article  Google Scholar 

  • Lewis, D. D., Yang, Y., Rose, T. G., & Li, F. (2004). Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5(Apr), 361–397.

  • Lewis, R. L., & Vasishth, S. (2005). An activation-based model of sentence processing as skilled memory retrieval. Cognitive Science, 29(3), 375–419.

    Article  Google Scholar 

  • Libben, G. (2005). Everything is psycholinguistics: Material and methodological considerations in the study of compound processing. Canadian Journal of Linguistics/Revue Canadienne de Linguistique, 50(1–4), 267–283.

    Article  Google Scholar 

  • Marzi, C., & Pirrelli, V. (2015). A neuro-computational approach to understanding the mental lexicon. Journal of Cognitive Science, 16(4), 493–534.

    Google Scholar 

  • Matsuki, K., Chow, T., Hare, M., Elman, J. L., Scheepers, C., & McRae, K. (2011). Event-based plausibility immediately influences on-line language comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(4), 913.

    Google Scholar 

  • McElree, B., Traxler, M. J., Pickering, M. J., Seely, R. E., & Jackendoff, R. (2001). Reading time evidence for enriched composition. Cognition, 78, B17–B25.

    Article  Google Scholar 

  • McRae, K., & Matsuki, K. (2009). People use their knowledge of common events to understand language, and do so as quickly as possible. Language and Linguistics Compass, 3(6), 1417–1429.

    Article  Google Scholar 

  • McRae, K., Spivey-Knowlton, M., & Tanenhaus, M. (1998). Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38(3), 283–312.

    Article  Google Scholar 

  • McRae, K., Hare, M., Elman, J. L., & Ferretti, T. (2005). A basis for generating expectancies for verbs from nouns. Memory and Cognition, 33(7), 1174–1184.

    Article  Google Scholar 

  • Mitchell, J., & Lapata, M. (2010). Composition in distributional models of semantics. Cognitive Science, 34(8), 1388–1429.

    Article  Google Scholar 

  • Pustejovsky, J., & Batiukova, O. (2019). The lexicon. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Resnik, P. (1997). Selectional preference and sense disambiguation. In Tagging text with lexical semantics: Why, what, and how?

  • Rimell, L., Maillard, J., Polajnar, T., & Clark, S. (2016). RELPRON: A relative clause evaluation dataset for compositional distributional semantics. Computational Linguistics, 42(4), 661–701.

    Article  Google Scholar 

  • Roth, M., & Lapata, M. (2015). Context-aware frame-semantic role labeling. Transactions of the Association for Computational Linguistics, 3, 449–460.

    Article  Google Scholar 

  • Santus, E., Chersoni, E., Lenci, A., & Blache, P. (2017). Measuring thematic fit with distributional feature overlap. In Proceedings of EMNLP.

  • Sayeed, A., Demberg, V., & Shkadzko, P. (2015). An exploration of semantic features in an unsupervised thematic fit evaluation framework. Italian Journal of Computational Linguistics, 1(1), 31–46.

    Article  Google Scholar 

  • Tilk, O., Demberg, V., Sayeed, A., Klakow, D., & Thater, S. (2016). Event participant modelling with neural networks. In Proceedings of the EMNLP.

  • Traxler, M. J., Pickering, M. J., & McElree, B. (2002). Coercion in sentence processing: Evidence from eye-movements and self-paced reading. Journal of Memory and Language, 47(4), 530–547.

    Article  Google Scholar 

  • Vassallo, P., Chersoni, E., Santus, E., Lenci, A., & Blache, P. (2018). Event knowledge in sentence processing: A new dataset for the evaluation of argument typicality. In Proceedings of the LREC workshop on linguistic and neuro-cognitive resources (LiNCR).

  • Zapirain, B., Agirre, E., Marquez, L., & Surdeanu, M. (2013). Selectional preferences for semantic role classification. Computational Linguistics, 39(3), 631–663.

    Article  Google Scholar 

  • Zarcone, A., Utt, J., & Padó, S. (2012). Modeling covert event retrieval in logical metonymy: Probabilistic and distributional accounts. In Proceedings of the NAACL workshop on cognitive modeling and computational linguistics.

  • Zarcone, A., Lenci, A., Padó, S., & Utt, J. (2013). Fitting, not clashing! A distributional semantic model of logical metonymy. In Proceedings of IWCS.

  • Zarcone, A., Padó, S., & Lenci, A. (2014). Logical metonymy resolution in a words-as-cues framework: Evidence from self-paced reading and probe recognition. Cognitive Science, 38(5), 973–996.

    Article  Google Scholar 

  • Zhang, H., Ding, H., & Song, Y. (2019). SP-10K: A large-scale evaluation set for selectional preference acquisition. In Proceedings of ACL.

  • Zhang, H., Bai, J., Song, Y., Xu, K., Yu, C., Song, Y., Ng, W., & Yu, D. (2020.) Multiplex word embeddings for selectional preference acquisition. arXiv preprint arXiv:200102836.

Download references

Author information

Authors and Affiliations

  1. Hong Kong Polytechnic University, Hung Hom, Hong Kong

    Emmanuele Chersoni & Chu-Ren Huang

  2. Massachusetts Institute of Technology, Cambridge, USA

    Enrico Santus

  3. University of Pisa, Pisa, Italy

    Alessandro Lenci

  4. Aix-Marseille University, Marseille, France

    Philippe Blache

Authors
  1. Emmanuele Chersoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enrico Santus
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessandro Lenci
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philippe Blache
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chu-Ren Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emmanuele Chersoni.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chersoni, E., Santus, E., Lenci, A. et al. Not all arguments are processed equally: a distributional model of argument complexity. Lang Resources & Evaluation 55, 873–900 (2021). https://doi.org/10.1007/s10579-021-09533-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10579-021-09533-9

Keywords

Search

Navigation