揭示余弦相似度在什么情况下有效、何时会失效
Semantics at an Angle: When Cosine Similarity Works Until It Doesn't
- 从几何角度分析余弦相似度的适用条件与局限
- 指出归一化会丢失径向信息,导致相似度误判
- 适合研究嵌入表示、相似度度量的学者参考
余弦相似度是信息检索、自然语言处理、计算机视觉和多模态学习中常用的学习表征比较方法。其流行有充分理由:它消除正向径向尺度影响,计算便捷,且常与归一化嵌入训练目标一致。但这些特性也限制了余弦相似度的表达能力:归一化会丢弃径向变化;各向异性表征可能压缩角度对比;高维邻域易形成中心节点(hub);对称且未校准的得分可能与实际关系不匹配。本文围绕一个核心原则展开:余弦相似度的有效性取决于学习表征、归一化或后处理方式、评分规则以及下游决策任务。我们推导主要几何恒等式,区分常被混淆的失败机制,回顾嵌入范数的相关证据,并介绍目标匹配、几何感知、抗中心节点、范数感知及可学习的替代方案。核心结论是条件性的而非对抗性的:当径向变化为干扰项或由表示约定固定时,余弦相似度的尺度不变性合理,但仍需验证其角度结构与下游决策是否匹配。
原文摘要 · Abstract (English)
Cosine similarity is a standard comparison rule for learned representations in information retrieval, natural language processing, computer vision, and multimodal learning. Its popularity is well founded: it removes positive radial scale, is computationally convenient, and often matches objectives that train normalized embeddings. These same properties also delimit what cosine can express. Normalization discards radial variation; anisotropic representations can compress angular contrast; high-dimensional neighborhoods can develop hubs; and a symmetric, uncalibrated score may be mismatched to the relation of interest. This article offers a selective review organized around a simple principle: the usefulness of cosine similarity depends jointly on the learned representation, any normalization or post-processing, the scoring rule, and the downstream decision. We derive the main geometric identities, distinguish failure mechanisms that are often conflated, review representative evidence about embedding norms, and describe objective-matched, geometry-aware, hubness-aware, norm-aware, and learned alternatives. The central conclusion is conditional rather than adversarial: cosine's positive-scale invariance is justified when radial variation is nuisance or fixed by the representation contract, but its angular geometry and downstream decision must still be validated.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。