Temporary syntactic ambiguities arise when the beginning of a sentence is compatible with multiple syntactic analyses. We inspect to which extent neural language models (LMs) exhibit uncertainty over such analyses when processing temporarily ambiguous inputs, and how that uncertainty is modulated by disambiguating cues. We probe the LM's expectations by generating from it: we use stochastic decoding to derive a set of sentence completions, and estimate the probability that the LM assigns to each interpretation based on the distribution of parses across completions. Unlike scoring-based methods for targeted syntactic evaluation, this technique makes it possible to explore completions that are not hypothesized in advance by the researcher. We apply this method to study the behavior of two LMs (GPT2 and an LSTM) on three types of temporary ambiguity, using materials from human sentence processing experiments. We find that LMs can track multiple analyses simultaneously; the degree of uncertainty varies across constructions and contexts. As a response to disambiguating cues, the LMs often select the correct interpretation, but occasional errors point to potential areas of improvement.

中文翻译：

---

理解提示的语言模型是模棱两可的：通过生成探索句法不确定性

当句子的开头与多种句法分析兼容时，就会出现暂时的句法歧义。我们检查了神经语言模型 (LM) 在处理临时模糊输入时对此类分析表现出的不确定性的程度，以及如何通过消除歧义线索来调节这种不确定性。我们通过从中生成来探测 LM 的期望：我们使用随机解码来导出一组句子完成，并根据完成的解析分布估计 LM 分配给每个解释的概率。与用于目标句法评估的基于评分的方法不同，这种技术可以探索研究人员事先没有假设的完成。我们使用来自人类句子处理实验的材料，应用这种方法来研究两个 LM（GPT2 和 LSTM）在三种类型的临时歧义上的行为。我们发现 LM 可以同时跟踪多个分析；不确定性的程度因结构和环境而异。作为对消除歧义的线索的回应，LM 通常选择正确的解释，但偶尔的错误指向潜在的改进领域。