发现词向量差值在特定投影下呈现螺旋状几何结构
A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings
- 通过分析反义词与同义词的向量差值几何特征
- 在多种模型中观察到一致的螺旋形投影模式
- 适合对词向量空间几何感兴趣的自然语言处理研究者
反义词常被定义为具有所有相同语境属性但一个例外的词对。鉴于变压器模型似乎将概念编码为方向，这引发了一个问题：能否通过词对嵌入向量的几何特性，特别是其差值向量，检测出‘反义性’？此类几何研究自然需要与同义词进行对比。本文最初是一项探索性项目，旨在研究检测反义词对嵌入向量几何结构所需的系统复杂性。我们报告的现象是在某种特定投影配置下，多个嵌入模型中出现了一种奇特的‘漩涡’结构。
原文摘要 · Abstract (English)
Antonyms, or opposites, are sometimes defined as \emph{word pairs that have all of the same contextually relevant properties but one}. Seeing how transformer models seem to encode concepts as directions, this begs the question if one can detect ``antonymity'' in the geometry of the embedding vectors of word pairs, especially based on their difference vectors. Such geometrical studies are then naturally contrasted by comparing antonymic pairs to their opposites; synonyms. This paper started as an exploratory project on the complexity of the systems needed to detect the geometry of the embedding vectors of antonymic word pairs. What we now report is a curious ``swirl'' that appears across embedding models in a somewhat specific projection configuration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。