Weighted finite automata (WFA) are often used to represent probabilistic models, such as $n$-gram language models, since they are efficient for recognition tasks in time and space. The probabilistic source to be represented as a WFA, however, may come in many forms. Given a generic probabilistic model over sequences, we propose an algorithm to approximate it as a weighted finite automaton such that the Kullback-Leiber divergence between the source model and the WFA target model is minimized. The proposed algorithm involves a counting step and a difference of convex optimization step, both of which can be performed efficiently. We demonstrate the usefulness of our approach on various tasks, including distilling $n$-gram models from neural models, building compact language models, and building open-vocabulary character models. The algorithms used for these experiments are available in an open-source software library.

中文翻译：

---

将概率模型近似为加权有限自动机

加权有限自动机 (WFA) 通常用于表示概率模型，例如 $n$-gram 语言模型，因为它们对于时间和空间上的识别任务都很有效。然而，要表示为 WFA 的概率源可能有多种形式。给定序列上的通用概率模型，我们提出了一种算法来将其近似为加权有限自动机，从而使源模型和 WFA 目标模型之间的 Kullback-Leiber 分歧最小化。所提出的算法包括一个计数步骤和一个差异凸优化步骤，这两个步骤都可以有效地执行。我们展示了我们的方法在各种任务上的有用性，包括从神经模型中提取 $n$-gram 模型、构建紧凑语言模型和构建开放词汇字符模型。