Memory-augmented neural networks equip a recurrent neural network with an explicit memory to support tasks that require information storage without interference over long times. A key motivation for such research is to perform classic computation tasks, such as parsing. However, memory-augmented neural networks are notoriously hard to train, requiring many backpropagation epochs and a lot of data. In this paper, we introduce the reservoir stack machine, a model which can provably recognize all deterministic context-free languages and circumvents the training problem by training only the output layer of a recurrent net and employing auxiliary information during training about the desired interaction with a stack. In our experiments, we validate the reservoir stack machine against deep and shallow networks from the literature on three benchmark tasks for Neural Turing machines and six deterministic context-free languages. Our results show that the reservoir stack machine achieves zero error, even on test sequences longer than the training data, requiring only a few seconds of training time and 100 training sequences.

中文翻译：

---

储层堆垛机

内存增强型神经网络为循环神经网络配备了显式内存，以支持需要信息存储而不会长时间干扰的任务。进行此类研究的主要动机是执行经典的计算任务，例如解析。然而，众所周知，增强记忆的神经网络很难训练，需要许多反向传播时期和大量数据。在本文中，我们介绍了一种油藏堆栈机，该模型可以证明可识别所有上下文无关的语言，并通过仅训练递归网络的输出层并在训练过程中使用辅助信息与期望的交互作用来辅助训练信息，从而规避了训练问题。堆。在我们的实验中 我们根据有关神经图灵机的三种基准任务和六种确定性上下文无关语言的文献对深层和浅层网络进行了验证，从而验证了油藏堆栈机的适用性。我们的结果表明，即使在比训练数据更长的测试序列上，油藏堆栈机也实现了零误差，只需要几秒钟的训练时间和100个训练序列。