提出推理时校准注意力，让长文本中后段信息更易被检索到。
Attention Calibration for Position-Fair Dense Information Retrieval
- 在不重训练前提下，通过调节注意力分布减少位置偏差。
- 部分校准（λ=0.5）比全校准效果更好，提升多模型平均检索性能。
- 配置通用性强，适用于多种语言、领域和模型结构，适合部署优化。
密集检索模型存在位置偏差：当相关答案出现在段落后部时，检索效果下降（Zeng et al., 2025）。本文探讨是否可在推理阶段消除该偏差，无需重新训练且不牺牲整体检索性能。我们引入推理时注意力校准（Schuhmacher et al., 2026），并扩展引入强度系数λ，实现原始与完全校准注意力的插值。在SQuAD-PosQ与FineWeb-PosQ上，评估了篮子大小（B=128）、校准层范围（50%深度）及λ值对位置公平性与检索效果权衡的影响。结果显示，部分校准常优于全校准。一个统一配置（B=128, λ=0.5, 50%层深）在所有三类嵌入模型上均提升FineWeb-PosQ的nDCG@10调和平均，且无需每模型调参；该配置同时适用于<s>-pooling与last-token-pooling架构。该默认设置迁移至跨10语言、31领域的PosIR，在全部16种长度四分位×模型×检索设置组合中降低位置敏感指数，同时保持或提升整体nDCG@10。代码已开源：https://github.com/impresso/fair-sentence-transformers
原文摘要 · Abstract (English)
Dense retrieval models exhibit positional bias: retrieval effectiveness degrades when relevant information appears later in a passage (Zeng et al., 2025). We ask whether this bias can be reduced at inference time, without retraining and without sacrificing overall retrieval effectiveness. To this end, we adapt inference-time attention calibration (Schuhmacher et al., 2026) to downstream retrieval and extend it with a strength coefficient lambda that interpolates between the original and fully calibrated attention distributions. Across three embedding models on SQuAD-PosQ and FineWeb-PosQ, we examine how basket size, calibrated layer set, and strength affect the trade-off between positional fairness and retrieval effectiveness, finding that partial calibration frequently outperforms full calibration. A single configuration (B=128, lambda=0.5, 50% layer depth) improves the harmonic mean of nDCG@10 across positional groups on FineWeb-PosQ for all three models without per-model tuning, and applies to both <s>-pooled and last-token-pooled architectures. This default configuration transfers without modification to PosIR, which spans 10 languages and 31 domains, reducing the Position Sensitivity Index in all 16 length-quartile x model x retrieval-setting combinations, while preserving or improving aggregate nDCG@10. We release our extended codebase at https://github.com/impresso/fair-sentence-transformers
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。