通过消除文本偏好，让模型更依赖视觉信息回答多选题。
BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA
- 用文本单独预测结果修正多模态输出，避免语言干扰。
- 在5个数据集上平均准确率领先现有方法，推理速度提升28%以上。
- 适合追求高效、鲁棒多模态推理的研究者和应用开发者。
多模态多项选择题问答（MCQA）为评估视觉语言模型（VLMs）提供了标准化且可量化的评测场景。然而，由于候选选项被纳入输入上下文，该格式引入了多种非预期偏差。以往工作主要关注结构化偏差，如对特定选项的偏好。本文认为，候选选项充当了文本先验，导致模型更倾向于语言上合理的选项，而非依赖视觉内容。我们提出并实证验证：当多模态分布与纯文本分布显著偏离时，模型才真正依赖视觉证据。基于此，我们提出BUZZY——一种无需训练的解码方法，通过从多模态预测中减去纯文本分布来纠正结果。在五个VLM和五个多模态MCQA基准上的实验表明，BUZZY在当前最先进方法中达到最高平均准确率，同时相比先前对比解码方法推理延迟降低超28%。结果表明，通过抑制纯文本偏好以增强视觉信号，是实现高效且稳健多模态推理的关键。
原文摘要 · Abstract (English)
Multimodal multiple-choice question answering (MCQA) provides a standardized and objectively measurable setting for evaluating vision-language models (VLMs). However, because the MCQA format incorporates the candidate choices into the input context, it introduces several unintended biases. Previous work has primarily focused on structural biases, such as preferences for certain choices. Instead, we argue that the choices act as textual priors, causing models to favor linguistically plausible options regardless of the visual content. We hypothesize and empirically verify that a model genuinely relies on visual evidence only when its multimodal distribution significantly diverges from its text-only distribution. Based on this observation we propose BUZZY,a training-free decoding method that corrects multimodal predictions by subtracting the text-only distribution. Experiments with five VLMs on five multimodal MCQA benchmarks demonstrate that BUZZY achieves the highest average accuracy among state-of-the-art methods while reducing inference latency by over 28% compared to prior contrastive decoding approaches. Overall, these results suggest that amplifying the visual signal by penalizing text-only preferences is key to efficient and robust multimodal MCQA reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。