当前位置:
X-MOL 学术
›
arXiv.cs.FL
›
论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Extracting Weighted Automata for Approximate Minimization in Language Modelling
arXiv - CS - Formal Languages and Automata Theory Pub Date : 2021-06-05 , DOI: arxiv-2106.02965 Clara Lacroce, Prakash Panangaden, Guillaume Rabusseau
arXiv - CS - Formal Languages and Automata Theory Pub Date : 2021-06-05 , DOI: arxiv-2106.02965 Clara Lacroce, Prakash Panangaden, Guillaume Rabusseau
In this paper we study the approximate minimization problem for language
modelling. We assume we are given some language model as a black box. The
objective is to obtain a weighted finite automaton (WFA) that fits within a
given size constraint and which mimics the behaviour of the original model
while minimizing some notion of distance between the black box and the
extracted WFA. We provide an algorithm for the approximate minimization of
black boxes trained for language modelling of sequential data over a one-letter
alphabet. By reformulating the problem in terms of Hankel matrices, we leverage
classical results on the approximation of Hankel operators, namely the
celebrated Adamyan-Arov-Krein (AAK) theory. This allows us to use the spectral
norm to measure the distance between the black box and the WFA. We provide
theoretical guarantees to study the potentially infinite-rank Hankel matrix of
the black box, without accessing the training data, and we prove that our
method returns an asymptotically-optimal approximation.
中文翻译:
为语言建模中的近似最小化提取加权自动机
在本文中,我们研究了语言建模的近似最小化问题。我们假设给定了一些语言模型作为黑匣子。目标是获得一个加权有限自动机 (WFA),它适合给定的大小约束,并模仿原始模型的行为,同时最小化黑盒和提取的 WFA 之间的距离概念。我们提供了一种近似最小化黑盒的算法,用于对单字母字母表上的顺序数据进行语言建模。通过根据 Hankel 矩阵重新表述问题,我们利用 Hankel 算子近似的经典结果,即著名的 Adamyan-Arov-Krein (AAK) 理论。这允许我们使用谱范数来测量黑匣子和 WFA 之间的距离。
更新日期:2021-06-08
中文翻译:
为语言建模中的近似最小化提取加权自动机
在本文中,我们研究了语言建模的近似最小化问题。我们假设给定了一些语言模型作为黑匣子。目标是获得一个加权有限自动机 (WFA),它适合给定的大小约束,并模仿原始模型的行为,同时最小化黑盒和提取的 WFA 之间的距离概念。我们提供了一种近似最小化黑盒的算法,用于对单字母字母表上的顺序数据进行语言建模。通过根据 Hankel 矩阵重新表述问题,我们利用 Hankel 算子近似的经典结果,即著名的 Adamyan-Arov-Krein (AAK) 理论。这允许我们使用谱范数来测量黑匣子和 WFA 之间的距离。