当前位置: X-MOL 学术arXiv.cs.LG › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Alignment and stability of embeddings: measurement and inference improvement
arXiv - CS - Machine Learning Pub Date : 2021-01-18 , DOI: arxiv-2101.07251
Furkan Gürsoy, Mounir Haddad, Cécile Bothorel

Representation learning (RL) methods learn objects' latent embeddings where information is preserved by distances. Since distances are invariant to certain linear transformations, one may obtain different embeddings while preserving the same information. In dynamic systems, a temporal difference in embeddings may be explained by the stability of the system or by the misalignment of embeddings due to arbitrary transformations. In the literature, embedding alignment has not been defined formally, explored theoretically, or analyzed empirically. Here, we explore the embedding alignment and its parts, provide the first formal definitions, propose novel metrics to measure alignment and stability, and show their suitability through synthetic experiments. Real-world experiments show that both static and dynamic RL methods are prone to produce misaligned embeddings and such misalignment worsens the performance of dynamic network inference tasks. By ensuring alignment, the prediction accuracy raises by up to 90% in static and by 40% in dynamic RL methods.

中文翻译:

嵌入的对齐方式和稳定性:测量和推理改进

表示学习(RL)方法学习对象的潜在嵌入,其中信息通过距离保存。由于距离对于某些线性变换是不变的,因此在保留相同信息的同时可以获得不同的嵌入。在动态系统中,嵌入的时间差异可以通过系统的稳定性或由于任意变换导致的嵌入未对齐来解释。在文献中,尚未对嵌入对齐方式进行正式定义,理论探讨或经验分析。在这里,我们探讨了嵌入比对及其部分,提供了第一个正式定义,提出了用于测量比对和稳定性的新颖指标,并通过合成实验证明了它们的适用性。实际实验表明,静态和动态RL方法都容易产生未对齐的嵌入,并且这种不对齐会使动态网络推理任务的性能恶化。通过确保对齐,在静态RL方法中,预测精度最多可提高90%,在动态RL方法中,则可提高40%。
更新日期:2021-01-19
down
wechat
bug