发现大模型在多选题中偏好特定选项，提出无需重训练的修正方法。
Benchmarking and Mitigating MCQA Selection Bias of Large Vision-Language Models
- 构建细粒度多选题基准，按选项语义相似度分难易
- 模型在难题中更倾向选靠前选项，偏差随难度上升
- 推理时动态校正输出，提升高难度任务准确率
大型视觉语言模型（LVLMs）在视觉问答（VQA）任务上表现强劲，但多选题回答（MCQA）中的选择偏差问题——即模型倾向于特定选项标记（如"A"）或位置——尚未被充分研究。本文通过覆盖简单、中等和困难难度的细粒度MCQA基准，系统考察了该偏差的存在与特性，难度由选项间语义相似度定义。我们提出一种推理阶段的对数空间去偏方法：从通用与上下文提示中估计集成偏差向量，并基于置信度自适应地修正模型输出。该方法无需重新训练，兼容冻结的LVLM。跨多个前沿模型的大量实验表明，选择偏差普遍存在且随任务难度加剧；所提方法显著降低偏差并提升复杂场景下的准确率。本工作揭示了LVLM在细粒度视觉推理中的局限性，并提供了一种实用的增强鲁棒性方案。数据集与代码见：https://github.com/Atabuzzaman/Selection-Bias-of-LVLMs
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) have achieved strong performance on vision-language tasks, particularly Visual Question Answering (VQA). While prior work has explored unimodal biases in VQA, the problem of selection bias in Multiple-Choice Question Answering (MCQA), where models may favor specific option tokens (e.g., "A") or positions, remains underexplored. In this paper, we investigate both the presence and nature of selection bias in LVLMs through fine-grained MCQA benchmarks spanning easy, medium, and hard difficulty levels, defined by the semantic similarity of the options. We further propose an inference-time logit-level debiasing method that estimates an ensemble bias vector from general and contextual prompts and applies confidence-adaptive corrections to the model's output. Our method mitigates bias without retraining and is compatible with frozen LVLMs. Extensive experiments across several state-of-the-art models reveal consistent selection biases that intensify with task difficulty, and show that our mitigation approach significantly reduces bias while improving accuracy in challenging settings. This work offers new insights into the limitations of LVLMs in MCQA and presents a practical approach to improve their robustness in fine-grained visual reasoning. Datasets and code are available at: https://github.com/Atabuzzaman/Selection-Bias-of-LVLMs
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。