纠正误解：归一化后余弦相似度无歧义，与欧氏距离等价
In Defense of Cosine Similarity: Normalization Eliminates the Gauge Freedom
- 将嵌入向量约束在单位球面，消除对角缩放带来的歧义
- 归一化后余弦距离恒等于欧氏距离的一半，排名完全一致
- 适合做向量相似性计算的研究者，尤其关注嵌入归一化的场景
Steck、Ekanadham 与 Kallus [arXiv:2403.05440] 证明，矩阵分解模型学习到的嵌入向量的余弦相似度可被对角“规范”矩阵 $D$ 任意改变。该结论正确且对使用点积目标训练的嵌入计算余弦相似度的实践者具有重要警示意义。然而我们主张，其结论——普遍警惕余弦相似度——混淆了不兼容训练目标所导致的病理现象与余弦距离在单位球面上的几何有效性。我们证明，当嵌入向量被约束在单位球面 $\mathbb{S}^{d-1}$（训练中或训练后通过合适目标实现）时，$D$ 矩阵的歧义完全消失，余弦距离精确等价于平方欧氏距离的一半。这一单调等价关系意味着，在归一化嵌入上，基于余弦和基于欧氏的距离排名完全一致。余弦相似度的问题不在于余弦本身，而在于未进行归一化。
原文摘要 · Abstract (English)
Steck, Ekanadham, and Kallus [arXiv:2403.05440] demonstrate that cosine similarity of learned embeddings from matrix factorization models can be rendered arbitrary by a diagonal ``gauge'' matrix $D$. Their result is correct and important for practitioners who compute cosine similarity on embeddings trained with dot-product objectives. However, we argue that their conclusion, cautioning against cosine similarity in general, conflates the pathology of an incompatible training objective with the geometric validity of cosine distance on the unit sphere. We prove that when embeddings are constrained to the unit sphere $\mathbb{S}^{d-1}$ (either during or after training with an appropriate objective), the $D$-matrix ambiguity vanishes identically, and cosine distance reduces to exactly half the squared Euclidean distance. This monotonic equivalence implies that cosine-based and Euclidean-based neighbor rankings are identical on normalized embeddings. The ``problem'' with cosine similarity is not cosine similarity, it is the failure to normalize.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。