This paper takes a first step towards a critical thinking curriculum for neural auto-regressive language models. We introduce a synthetic text corpus of deductively valid arguments, and use this artificial argument corpus to train and evaluate GPT-2. Significant transfer learning effects can be observed: Training a model on a few simple core schemes allows it to accurately complete conclusions of different, and more complex types of arguments, too. The language models seem to connect and generalize the core argument schemes in a correct way. Moreover, we obtain consistent and promising results for the GLUE and SNLI benchmarks. The findings suggest that there might exist a representative sample of paradigmatic instances of good reasoning that will suffice to acquire general reasoning skills and that might form the core of a critical thinking curriculum for language models.

中文翻译：

---

语言模型的批判性思维

本文向神经自回归语言模型的批判性思维课程迈出了第一步。我们引入了一个演绎有效论证的合成文本语料库，并使用这个人工论证语料库来训练和评估 GPT-2。可以观察到显着的迁移学习效果：在几个简单的核心方案上训练模型也可以使其准确地完成不同类型和更复杂的论点的结论。语言模型似乎以正确的方式连接和概括了核心参数方案。此外，我们在 GLUE 和 SNLI 基准测试中获得了一致且有希望的结果。