对比解码看似提升性能，实则靠误导性机制，无法真正减少幻觉。
The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs?
- 通过构造反例诱导幻觉并抑制，但实际效果被误导因素掩盖。
- 在POPE基准上表现提升，实因输出分布单向调整和自适应约束。
- 揭示现有方法本质是贪心搜索，适合研究幻觉机制的学者参考。
对比解码策略广泛用于减少多模态大语言模型（MLLMs）中的对象幻觉。这些方法通过构建对比样本诱导幻觉，并在输出分布中抑制它们。然而，本文表明此类方法无法有效缓解幻觉问题。在POPE基准上观察到的性能提升，主要由两个误导性因素驱动：(1) 对模型输出分布的粗粒度、单向调整；(2) 自适应可接受性约束，使采样策略退化为贪心搜索。为进一步说明问题，我们引入一系列伪提升方法，并与对比解码技术进行对比评估。实验结果表明，对比解码所呈现的性能增益与其缓解幻觉的初衷完全无关。研究挑战了关于对比解码有效性的一般假设，为开发真正有效的幻觉解决方案铺平道路。
原文摘要 · Abstract (English)
Contrastive decoding strategies are widely used to reduce object hallucinations in multimodal large language models (MLLMs). These methods work by constructing contrastive samples to induce hallucinations and then suppressing them in the output distribution. However, this paper demonstrates that such approaches fail to effectively mitigate the hallucination problem. The performance improvements observed on POPE Benchmark are largely driven by two misleading factors: (1) crude, unidirectional adjustments to the model's output distribution and (2) the adaptive plausibility constraint, which reduces the sampling strategy to greedy search. To further illustrate these issues, we introduce a series of spurious improvement methods and evaluate their performance against contrastive decoding techniques. Experimental results reveal that the observed performance gains in contrastive decoding are entirely unrelated to its intended goal of mitigating hallucinations. Our findings challenge common assumptions about the effectiveness of contrastive decoding strategies and pave the way for developing genuinely effective solutions to hallucinations in MLLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。