arXiv:2602.09229cs.LGcs.IR2026-02被引 1

提出新框架,发现嵌入大小在检索中影响得分与梯度,可提升性能达72%。

When Does Embedding Magnitude Matter? A Cross-Task Functional-Symmetry Framework

  • 分控查询和文档侧归一化,发现单边归一化更优
  • 跨任务测试显示最大提升达72%,下游RAG提升24%
  • 适用于推荐、少样本分类等非对称任务

余弦相似度对两侧都归一化,点积则都不归一。我们提出一个2×2框架,独立控制查询侧和文档侧的归一化,揭示了此前未被研究的两种中间变体(QNorm、DNorm)。在四个编码器上评估,于MS MARCO域内及BEIR、BRIGHT和多跳问答域外测试中,单边归一化优于余弦和点积,域外相对提升最高达+72%,下游RAG任务提升+24%。交叉验证表明:文档嵌入大小影响推理得分,查询嵌入大小调节训练梯度,且费舍尔信息矩阵条件数可预测应归一化哪一侧。我们进一步按函数对称性分类任务——即评分过程是否将查询与文档视为可互换——并检验该机制是否超越检索。在五类额外任务(语义文本相似度、CLIP、知识图谱补全、少样本分类、推荐系统)中,粗略预测(对称任务用余弦,非对称任务保留大小)全部成立;单边变体在推荐中优于余弦,在少样本分类中DNorm优于余弦和原型网络的标准欧氏默认。

原文摘要 · Abstract (English)

Cosine similarity normalizes both sides; dot product normalizes neither. We propose a 2x2 framework that independently controls query-side and document-side normalization, exposing two intermediate variants (QNorm, DNorm) that have not been previously studied. On retrieval with four encoders, evaluated in-domain on MS MARCO and out-of-domain on BEIR, BRIGHT, and multi-hop QA, the unilateral variants outperform both cosine and dot product, with relative gains of up to +72% out-of-domain and +24% on downstream RAG. Cross-evaluation reveals the mechanism: document magnitude scales inference scores while query magnitude modulates training gradients, and the Fisher Information Matrix condition number predicts which side to normalize. We then classify tasks by functional symmetry, defined as whether the aggregate scoring procedure treats Q and C as interchangeable, and test whether the mechanism extends beyond retrieval. On five additional task families (semantic textual similarity, CLIP, knowledge graph completion, few-shot classification, recommender systems), the coarse prediction (cosine for symmetric, magnitude-preserving for asymmetric) holds in every case examined; the unilateral variants beat Cosine on recommendation, and on few-shot classification DNorm beats both Cosine and the standard Euclidean default of Prototypical Networks.

嵌入归一化检索优化少样本学习推荐系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。