当前位置: X-MOL 学术arXiv.cs.FL › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Separation of Memory and Processing in Dual Recurrent Neural Networks
arXiv - CS - Formal Languages and Automata Theory Pub Date : 2020-05-17 , DOI: arxiv-2005.13971
Christian Oliva and Luis F. Lago-Fern\'andez

We explore a neural network architecture that stacks a recurrent layer and a feedforward layer that is also connected to the input, and compare it to standard Elman and LSTM architectures in terms of accuracy and interpretability. When noise is introduced into the activation function of the recurrent units, these neurons are forced into a binary activation regime that makes the networks behave much as finite automata. The resulting models are simpler, easier to interpret and get higher accuracy on different sample problems, including the recognition of regular languages, the computation of additions in different bases and the generation of arithmetic expressions.

中文翻译:

双循环神经网络中记忆和处理的分离

我们探索了一种神经网络架构,该架构堆叠了一个循环层和一个也连接到输入的前馈层,并在准确性和可解释性方面将其与标准 Elman 和 LSTM 架构进行了比较。当噪声被引入循环单元的激活函数时,这些神经元被迫进入二元激活机制,使网络表现得更像有限自动机。得到的模型更简单、更容易解释并在不同的样本问题上获得更高的准确率,包括正则语言的识别、不同基数的加法计算和算术表达式的生成。
更新日期:2020-05-29
down
wechat
bug