当前位置: X-MOL 学术arXiv.cs.FL › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Provably Stable Interpretable Encodings of Context Free Grammars in RNNs with a Differentiable Stack
arXiv - CS - Formal Languages and Automata Theory Pub Date : 2020-06-05 , DOI: arxiv-2006.03651
John Stogin, Ankur Mali and C Lee Giles

Given a collection of strings belonging to a context free grammar (CFG) and another collection of strings not belonging to the CFG, how might one infer the grammar? This is the problem of grammatical inference. Since CFGs are the languages recognized by pushdown automata (PDA), it suffices to determine the state transition rules and stack action rules of the corresponding PDA. An approach would be to train a recurrent neural network (RNN) to classify the sample data and attempt to extract these PDA rules. But neural networks are not a priori aware of the structure of a PDA and would likely require many samples to infer this structure. Furthermore, extracting the PDA rules from the RNN is nontrivial. We build a RNN specifically structured like a PDA, where weights correspond directly to the PDA rules. This requires a stack architecture that is somehow differentiable (to enable gradient-based learning) and stable (an unstable stack will show deteriorating performance with longer strings). We propose a stack architecture that is differentiable and that provably exhibits orbital stability. Using this stack, we construct a neural network that provably approximates a PDA for strings of arbitrary length. Moreover, our model and method of proof can easily be generalized to other state machines, such as a Turing Machine.

中文翻译:

具有可微堆栈的 RNN 中上下文无关文法的可证明稳定可解释编码

给定一组属于上下文无关文法 (CFG) 的字符串和另一组不属于 CFG 的字符串,如何推断文法?这就是语法推理的问题。由于CFG是下推自动机(PDA)识别的语言,因此确定相应PDA的状态转换规则和堆栈动作规则就足够了。一种方法是训练循环神经网络 (RNN) 来对样本数据进行分类并尝试提取这些 PDA 规则。但是神经网络并不能先验地了解 PDA 的结构,并且可能需要许多样本来推断这种结构。此外,从 RNN 中提取 PDA 规则并非易事。我们构建了一个特别像 PDA 结构的 RNN,其中权重直接对应于 PDA 规则。这需要在某种程度上可微分(以启用基于梯度的学习)和稳定(不稳定的堆栈将显示更长的字符串性能恶化)的堆栈架构。我们提出了一种可微且可证明具有轨道稳定性的堆栈架构。使用这个堆栈,我们构建了一个神经网络,该网络可证明近似于任意长度字符串的 PDA。此外,我们的模型和证明方法可以很容易地推广到其他状态机,例如图灵机。我们构建了一个神经网络,可以证明它近似于任意长度字符串的 PDA。此外,我们的模型和证明方法可以很容易地推广到其他状态机,例如图灵机。我们构建了一个神经网络,可以证明它近似于任意长度字符串的 PDA。此外,我们的模型和证明方法可以很容易地推广到其他状态机,例如图灵机。
更新日期:2020-06-12
down
wechat
bug