arXiv:2606.29571cs.CL2026-06

文本嵌入的相似性该用余弦还是排序度量?看几何分布。

Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings

论文配图:Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings
图 1 · 摘自论文原文
  • 根据嵌入空间的方差分布决定用哪种度量:均匀则用余弦,集中则用排序或L1类
  • 当主方向方差占比高时,非余弦度量平均提升20%,最强时效果更显著
  • 只需一个数(最大方向方差占比)就能预测度量优劣,适合工程部署诊断

比较文本嵌入的标准方法是余弦相似度。已有研究发现其他度量更优,但从未明确其几何条件。本文通过全面实证研究:在十九个编码器(从紧凑句向量模型到七亿参数大模型)上测试十九种无参相似度度量,覆盖七个数据集。结果表明:当编码器的方差在各方向上均匀分布时,余弦相似度为最优且其他度量无明显优势;当方差集中在少数主导方向(即各向异性)时,基于排序和L1类型的度量显著优于余弦,平均相对提升约20%。关键指标是单一维度方差占比,其与性能提升的相关性达秩相关0.86、线性相关0.95。通过投影移除主方向后,余弦恢复优势,仅在原本各向异性的模型中有效,证明效应源于方向性而非长度。因此,在广泛使用的微调嵌入器中,余弦仍是合适选择,本研究提供了一项一数值诊断工具。

原文摘要 · Abstract (English)

The standard way to compare two text embeddings is cosine similarity. Scattered studies report that a different metric does better, but never pin down the geometric condition that decides when, or why. We settle both with a comprehensive empirical study: nineteen parameter-free similarity metrics on nineteen encoders, from compact sentence transformers up to seven-billion-parameter large language models, across seven datasets. The answer is geometric. When an encoder spreads its variance evenly across directions, cosine is the best parameter-free choice and no other metric helps by a usable margin. When the variance concentrates into a few dominant directions, a property known as anisotropy, rank-based and L1-type metrics beat cosine by a clear margin. The absolute gain is modest, but because cosine starts low on these encoders it is a sizable relative improvement, around twenty percent on average and largest where cosine is weakest. What decides this is the geometry of the embedding space, not how the model was trained: where the two disagree, the metric follows the geometry. One number, the fraction of variance held by the single most dominant dimension, predicts how much the alternatives help across all nineteen encoders, with a rank correlation of 0.86 and a linear correlation of 0.95. To test this as the cause rather than a correlate, we project out the dominant directions: cosine recovers and the advantage of the other metrics nearly vanishes, but only on the encoders that were anisotropic to begin with. The effect is directional, not magnitude based, since it survives normalizing every vector to unit length. Among parameter-free metrics, then, cosine is the right tool wherever an encoder is well spread, which includes the fine-tuned embedders commonly deployed for retrieval, and we give a one-number diagnostic for when it is not.

嵌入空间相似度度量各向异性文本检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。