This work aims at shedding some light on connections between finite state machines (FSMs), and recurrent neural networks (RNNs). Examined connections in this master's thesis is threefold: the extractability of finite state machines from recurrent neural networks, learnability aspects and computationnal links. With respect to the former, the long-standing clustering hypothesis of RNN hidden state space when trained to recognize regular languages was explored, and new insights into this hypothesis through the lens of recent advances of the generalization theory of Deep Learning are provided. As for learnability, an extension of the active learning framework better suited to the problem of approximating RNNs with FSMs is proposed, with the aim of better formalizing the problem of RNN approximation by FSMs. Theoretical analysis of two possible scenarions in this framework were performed. With regard to computability, new computational results on the distance and the equivalence problem between RNNs trained as language models and different types of weighted finite state machines were given.

中文翻译：

---

循环神经网络中有限状态机的可计算性、可学习性和可提取性

这项工作旨在阐明有限状态机 (FSM) 和循环神经网络 (RNN) 之间的联系。本硕士论文中检查的连接有三个方面：从循环神经网络中提取有限状态机、可学习性方面和计算链接。对于前者，研究了 RNN 隐藏状态空间在训练识别常规语言时长期存在的聚类假设，并通过深度学习泛化理论的最新进展提供了对该假设的新见解。至于可学习性，提出了更适合用 FSM 逼近 RNN 问题的主动学习框架的扩展，目的是更好地将 FSM 逼近 RNN 的问题形式化。对该框架中的两种可能场景进行了理论分析。关于可计算性，给出了作为语言模型训练的 RNN 与不同类型的加权有限状态机之间的距离和等价问题的新计算结果。