用注意力信号纠正图像顺序偏差，让模型更关注内容而非位置。
Logit-Attention Divergence: Mitigating Position Bias in Multi-Image Retrieval via Attention-Guided Calibration
- 利用内部注意力图在推理时动态修正输出，无需额外训练。
- 在MS-COCO基准上准确率提升超40%，显著改善顺序无关性。
- 仅需少量校准数据，计算开销极低，适合实际部署。
多模态大语言模型在多图像跨模态检索中表现优异，但存在严重的输入位置偏差，即预测结果受图像顺序影响而非语义相关性。通过实证分析，我们发现一种称为「逻辑-注意力发散」的现象：尽管输出逻辑值严重偏倚，内部注意力图仍能良好对齐相关视觉证据。这一现象揭示了现有基于逻辑层校准方法（如PriDe）的根本局限。基于此，我们提出一种无需训练的、由注意力引导的去偏框架，在推理阶段利用内在注意力信号对实例级输出进行校准，仅需少量校准集且计算开销可忽略。在基于MS-COCO的基准测试中，该方法显著提升了排列不变性，达到当前最优性能，相较基线准确率提升超过40%。代码已公开于https://github.com/brightXian/LAD。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have shown strong performance in multi-image cross-modal retrieval, yet suffer from severe position bias, where predictions are dominated by input order rather than semantic relevance. Through empirical analysis, we identify a phenomenon termed Logit-Attention Divergence, in which output logits are heavily biased while internal attention maps remain well-aligned with relevant visual evidence. This observation reveals a fundamental limitation of existing logit-level calibration methods such as PriDe. Based on this insight, we propose a training-free, attention-guided debiasing framework that leverages intrinsic attention signals for instance-level correction at inference time, requiring only a minimal calibration set with negligible computational overhead. Experiments on MS-COCO-based benchmarks show that our method substantially improves permutation invariance and achieves state-of-the-art performance, enhancing accuracy by over 40\% compared to baselines. Code is available at https://github.com/brightXian/LAD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。