Several efforts aim at representing tabular data with neural models for supporting target applications at the intersection of natural language processing (NLP) and databases (DB) [1-3]. The goal is to extend to structured data the recent neural architectures, which achieve state of the art results in NLP applications. Language models (LMs) are usually pre-trained with unsupervised tasks on a large text corpus. The output LM is then fine-tuned on a variety of downstream tasks with a small set of specific examples. This process has many advantages, because the LM contains information about textual structure and content, which are used by the target application without manually defining features.

一些努力旨在用神经模型表示表格数据，以支持自然语言处理 (NLP) 和数据库 (DB) 交叉处的目标应用程序 [1-3]。目标是将最近的神经架构扩展到结构化数据，从而在 NLP 应用中实现最先进的结果。语言模型 (LM) 通常在大型文本语料库上使用无监督任务进行预训练。然后使用一小组特定示例在各种下游任务上对输出 LM 进行微调。这个过程有很多优点，因为 LM 包含有关文本结构和内容的信息，目标应用程序无需手动定义特征即可使用这些信息。