Recently, there has been a resurgence of formal language theory in deep learning research. However, most research focused on the more practical problems of attempting to represent symbolic knowledge by machine learning. In contrast, there has been limited research on exploring the fundamental connection between them. To obtain a better understanding of the internal structures of regular grammars and their corresponding complexity, we focus on categorizing regular grammars by using both theoretical analysis and empirical evidence. Specifically, motivated by the concentric ring representation, we relaxed the original order information and introduced an entropy metric for describing the complexity of different regular grammars. Based on the entropy metric, we categorized regular grammars into three disjoint subclasses: the polynomial, exponential and proportional classes. In addition, several classification theorems are provided for different representations of regular grammars. Our analysis was validated by examining the process of learning grammars with multiple recurrent neural networks. Our results show that as expected more complex grammars are generally more difficult to learn.

中文翻译：

---

使用循环神经网络进行规则语法分类和学习的熵度量

最近，形式语言理论在深度学习研究中重新兴起。然而，大多数研究都集中在尝试通过机器学习来表示符号知识的更实际问题上。相比之下，探索它们之间基本联系的研究有限。为了更好地理解正则文法的内部结构及其相应的复杂性，我们通过理论分析和经验证据对正则文法进行分类。具体来说，在同心环表示的推动下，我们放宽了原始顺序信息并引入了一个熵度量来描述不同正则文法的复杂性。基于熵度量，我们将正则文法分为三个不相交的子类：多项式、指数类和比例类。此外，还为正则文法的不同表示提供了几个分类定理。通过检查使用多个循环神经网络学习语法的过程，我们的分析得到了验证。我们的结果表明，正如预期的那样，更复杂的语法通常更难学习。