As modern deep networks become more complex, and get closer to human‐like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances—similar to human zero‐shot reasoning. However, we also note some shortcomings in this generalization behavior—similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human‐like language understanding.

中文翻译：

---

分析机器学习的表示：自然语言案例研究

随着现代深度网络变得越来越复杂，并且在某些领域越来越接近人类的能力，问题是它们学习的表征和决策规则与人类的相比如何。在这项工作中，我们研究了自然语言处理人工系统中句子的表示。我们首先提出一个诊断测试数据集来检查所表示的抽象可组合结构的程度。分析这些诊断测试的性能表明表示和决策规则缺乏系统性，并揭示了一组启发式策略。然后，我们研究了训练分布对学习这些启发式策略的影响，并研究了这些表示在训练集的各种增强下的变化。我们的结果揭示了与人类类似表示的相似之处。我们发现这些系统可以学习抽象规则，并在某些情况下将它们推广到新的上下文——类似于人类的零样本推理。然而，我们也注意到这种泛化行为的一些缺点——类似于人类的判断错误，如信念偏差。研究这些相似之处提出了理解人类心理现象的新方法，并为构建具有人类语言理解能力的人工智能提供了最佳策略。