Continuous representation of words is a standard component in deep learning-based NLP models. However, representing a large vocabulary requires significant memory, which can cause problems, particularly on resource-constrained platforms. Therefore, in this paper we propose an isotropic iterative quantization (IIQ) approach for compressing embedding vectors into binary ones, leveraging the iterative quantization technique well established for image retrieval, while satisfying the desired isotropic property of PMI based models. Experiments with pre-trained embeddings (i.e., GloVe and HDC) demonstrate a more than thirty-fold compression ratio with comparable and sometimes even improved performance over the original real-valued embedding vectors.

中文翻译：

---

使用各向同性迭代量化嵌入压缩

单词的连续表示是基于深度学习的 NLP 模型的标准组件。但是，表示大量词汇需要大量内存，这可能会导致问题，尤其是在资源受限的平台上。因此，在本文中，我们提出了一种各向同性迭代量化 (IIQ) 方法，用于将嵌入向量压缩为二进制向量，利用为图像检索而建立的迭代量化技术，同时满足基于 PMI 模型的所需各向同性特性。使用预训练嵌入（即 GloVe 和 HDC）进行的实验表明，压缩率提高了 30 倍以上，与原始实值嵌入向量相比，性能相当甚至有时甚至有所提高。