As a ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words, but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related to a predefined concept to take larger values along a specified dimension, while leaving the original semantic learning mechanism mostly unaffected. In other words, we align words that are already determined to be related, along predefined concepts. Therefore, we impart interpretability to the word embedding by assigning meaning to its vector dimensions. The predefined concepts are derived from an external lexical resource, which in this paper is chosen as Roget’s Thesaurus. We observe that alignment along the chosen concepts is not limited to words in the thesaurus and extends to other related words as well. We quantify the extent of interpretability and assignment of meaning from our experimental results. Manual human evaluation results have also been presented to further verify that the proposed method increases interpretability. We also demonstrate the preservation of semantic coherence of the resulting vector space using word-analogy/word-similarity tests and a downstream task. These tests show that the interpretability-imparted word embeddings that are obtained by the proposed framework do not sacrifice performances in common benchmark tests.

中文翻译：

---

在保留语义结构的同时赋予词嵌入可解释性

作为自然语言处理中普遍存在的方法，词嵌入被广泛用于将词的语义属性映射到密集的向量表示中。它们捕获单词之间的语义和句法关系，但与单词对应的向量仅相对于彼此有意义。向量及其维度都没有任何绝对的、可解释的含义。我们对嵌入学习算法的目标函数进行了附加修改，鼓励与预定义概念语义相关的词的嵌入向量沿指定维度取更大的值，而原始语义学习机制几乎不受影响。换句话说，我们将已经确定相关的单词与预定义的概念对齐。所以，我们通过为其向量维度分配含义来赋予词嵌入的可解释性。预定义的概念来源于外部词汇资源，在本文中被选为 Roget 的词库。我们观察到，沿所选概念的对齐不仅限于同义词库中的单词，还扩展到其他相关单词。我们从实验结果中量化了可解释性和意义分配的程度。还提供了人工人工评估结果，以进一步验证所提出的方法增加了可解释性。我们还使用词类比/词相似性测试和下游任务证明了结果向量空间的语义连贯性的保留。