ABSTRACT Whether neural networks can capture relational knowledge is a matter of long-standing controversy. Recently, some researchers have argued that (1) classic connectionist models can handle relational structure and (2) the success of deep learning approaches to natural language processing suggests that structured representations are unnecessary to model human language. We tested the Story Gestalt model, a classic connectionist model of text comprehension, and a Sequence-to-Sequence with Attention model, a modern deep learning architecture for natural language processing. Both models were trained to answer questions about stories based on abstract thematic roles. Two simulations varied the statistical structure of new stories while keeping their relational structure intact. The performance of each model fell below chance at least under one manipulation. We argue that both models fail our tests because they can't perform dynamic binding. These results cast doubts on the suitability of traditional neural networks for explaining relational reasoning and language processing phenomena.

中文翻译：

---

经典和现代语言处理神经网络模型的关系处理限制

摘要 神经网络能否捕获关系知识是一个长期存在争议的问题。最近，一些研究人员认为（1）经典联结主义模型可以处理关系结构，（2）自然语言处理深度学习方法的成功表明结构化表示对于模拟人类语言是不必要的。我们测试了 Story Gestalt 模型（一种经典的文本理解连接主义模型）和具有注意力机制的 Sequence-to-Sequence 模型（一种用于自然语言处理的现代深度学习架构）。两种模型都经过训练，可以根据抽象的主题角色回答有关故事的问题。两个模拟改变了新故事的统计结构，同时保持它们的关系结构完整。每个模型的性能至少在一次操作下低于机会。我们认为这两种模型都未能通过我们的测试，因为它们不能执行动态绑定。这些结果对传统神经网络在解释关系推理和语言处理现象方面的适用性提出了质疑。