分析晚交互模型的检索机制，发现其存在长度偏差与相似度分布问题。
Working Notes on Late Interaction Dynamics: Analyzing Targeted Behaviors of Late Interaction Models
- 研究多向量打分导致的长度偏差现象
- 验证最大相似度操作能有效利用词级相似度
- 适合关注检索模型内部机制的研究者阅读
尽管晚交互模型展现出强大的检索性能，但其底层动态机制仍缺乏深入研究，可能隐藏性能瓶颈。本文聚焦于晚交互检索中的两个问题：多向量打分引发的长度偏差，以及最大相似度算子合并最优分数后余下的相似度分布。我们在NanoBEIR基准上分析了当前最先进的模型。结果表明，因果型晚交互模型的理论长度偏差在实践中确实存在，而双向模型在极端情况下也可能遭受类似问题。此外，我们发现超越前1个文档词级别的相似度并无显著趋势，验证了MaxSim算子高效利用词级相似度的能力。
原文摘要 · Abstract (English)
While Late Interaction models exhibit strong retrieval performance, many of their underlying dynamics remain understudied, potentially hiding performance bottlenecks. In this work, we focus on two topics in Late Interaction retrieval: a length bias that arises when using multi-vector scoring, and the similarity distribution beyond the best scores pooled by the MaxSim operator. We analyze these behaviors for state-of-the-art models on the NanoBEIR benchmark. Results show that while the theoretical length bias of causal Late Interaction models holds in practice, bi-directional models can also suffer from it in extreme cases. We also note that no significant similarity trend lies beyond the top-1 document token, validating that the MaxSim operator efficiently exploits the token-level similarity scores.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。