A differentiable neural computer (DNC) is a memory augmented neural network devised to solve a wide range of algorithmic and question answering tasks and it showed promising performance in a variety of domains. However, its single memory-based operations are not enough to store and retrieve diverse informative representations existing in many tasks. Furthermore, DNC does not explicitly consider the memorization itself as a target objective, which inevitably leads to a very slow learning speed of the model. To address those issues, we propose a novel distributed memory-based self-supervised DNC architecture for enhanced memory augmented neural network performance. We introduce (i) a multiple distributed memory block mechanism that stores information independently to each memory block and uses stored information in a cooperative way for diverse representation and (ii) a self-supervised memory loss term which ensures how well a given input is written to the memory. Our experiments on algorithmic and question answering tasks show that the proposed model outperforms all other variations of DNC in a large margin, and also matches the performance of other state-of-the-art memory-based network models.

中文翻译：

---

基于分布式记忆的自监督可微神经计算机

可微分神经计算机 (DNC) 是一种记忆增强神经网络，旨在解决各种算法和问答任务，并在各种领域表现出良好的性能。然而，其单一的基于内存的操作不足以存储和检索存在于许多任务中的各种信息表示。此外，DNC 并没有明确地将记忆本身视为目标，这不可避免地导致模型的学习速度非常慢。为了解决这些问题，我们提出了一种新颖的基于分布式内存的自监督 DNC 架构，用于增强内存增强神经网络的性能。我们引入了 (i) 一种多分布式内存块机制，该机制独立地将信息存储到每个内存块，并以协作方式使用存储的信息进行多样化的表示，以及 (ii) 一个自监督的记忆丢失项，可确保给定输入的写入程度到内存。我们在算法和问答任务上的实验表明，所提出的模型在很大程度上优于 DNC 的所有其他变体，并且还与其他最先进的基于内存的网络模型的性能相匹配。