Abstract In their recent paper, Lau, Clark, and Lappin explore the idea that the probability of the occurrence of word strings can form the basis of an adequate theory of grammar (Lau, Jey H., Alexander Clark & 15 Shalom Lappin. 2017. Grammaticality, acceptability, and probability: A prob- abilistic view of linguistic knowledge. Cognitive Science 41(5):1201–1241). To make their case, they present the results of correlating the output of several probabilistic models trained solely on naturally occurring sentences with the gradient acceptability judgments that humans report for ungrammatical sentences derived from roundtrip machine translation errors. In this paper, we first explore the logic of the Lau et al. argument, both in terms of the choice of evaluation metric (gradient acceptability), and in the choice of test data set (machine translation errors on random sentences from a corpus). We then present our own series of studies intended to allow for a better comparison between LCL’s models and existing grammatical theories. We evaluate two of LCL’s probabilistic models (trigrams and recurrent neural network) against three data sets (taken from journal articles, a textbook, and Chomsky’s famous colorless-green-ideas sentence), using three evaluation metrics (LCL’s gradience metric, a categorical version of the metric, and the experimental-logic metric used in the syntax literature). Our results suggest there are very real, measurable cost-benefit tradeoffs inherent in LCL’s models across the three evaluation metrics. The gain in explanation of gradience (between 13% and 31% of gradience) is offset by losses in the other two metrics: a 43%-49% loss in coverage based on a categorical metric of explaining acceptability, and a loss of 12%-35% in explaining experimentally-defined phenomena. This suggests that anyone wishing to pursue LCL’s models as competitors with existing syntactic theories must either be satisfied with this tradeoff, or modify the models to capture the phenomena that are not currently captured.

中文翻译：

---

无色的绿色创意确实让我大失所望：渐变的可接受性和语法的本质

摘要在Lau，Clark和Lappin的最新论文中，他们探讨了单词字符串出现的概率可以构成充分语法理论的基础的观点（Lau，Jey H.，Alexander Clark和15 Shalom Lappin.2017。语法性，可接受性和可能性：语言知识的概率论（认知科学41（5）：1201–1241）。为了证明自己的观点，他们提出了将仅在自然发生的句子上训练的几种概率模型的输出与人类报告的从往返机器翻译错误中得出的非语法句子的梯度可接受性判断相关联的结果。在本文中，我们首先探讨Lau等人的逻辑。论据，无论是在评估指标的选择（梯度可接受性）方面，以及选择测试数据集（来自语料库的随机句子的机器翻译错误）。然后，我们提出自己的一系列研究，目的是为了更好地比较LCL的模型和现有的语法理论。我们使用三个评估指标（LCL的梯度度量，一种分类版本），针对三个数据集（取自期刊文章，一本教科书和Chomsky著名的无色绿色想法句子），针对三个LCL的概率模型（三元组和递归神经网络）进行评估。指标，以及语法文献中使用的实验逻辑指标）。我们的结果表明，在三个评估指标中，LCL模型具有内在的，非常实际的，可衡量的成本－收益权衡。解释梯度的收益（在梯度的13％到31％之间）被其他两个指标的损失所抵消：根据解释可接受性的分类指标，覆盖率损失了43％-49％，而在解释实验定义的现象时损失了12％-35％。这表明任何希望采用现有句法理论作为竞争对手的LCL模型的人都必须对此折衷感到满意，或者修改模型以捕获当前未捕获的现象。