提出解耦对比解码,有效抑制多模态大模型幻觉,不损失推理能力。
Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models
- 分离正负样本学习,用独立投影建模真实幻觉模式。
- 在多个幻觉基准上优于手工扰动方法,媲美训练型方案。
- 无需人工设计干扰,适合追求高可靠性的多模态应用。
尽管多模态大语言模型(MLLMs)在复杂多模态理解任务中表现出色,但仍存在显著的幻觉问题:生成内容与明显视觉或事实证据不符。现有基于训练的方案(如直接偏好优化,DPO)虽能抑制幻觉,但可能因概率分布偏移而牺牲通用推理能力。而无训练方案(如对比解码)通过从扭曲输入中减去估计的幻觉模式来实现目标,但其手工设计的扰动(如向图像加噪声)难以捕捉真实幻觉模式。为此,我们提出新框架——解耦对比解码(DCD)。DCD将偏好数据集中正负样本的学习解耦,在MLLM中分别训练正负图像投影。负投影隐式建模真实幻觉模式,使对比解码阶段可生成视觉感知的负样本图像。该方法避免成对优化,缓解概率偏移,无需手工降质,具备强泛化性。大量消融实验表明,DCD在幻觉检测基准和通用推理任务上均表现优异,既达到DPO的幻觉抑制效果,又保持通用能力,超越传统对比解码方法。
原文摘要 · Abstract (English)
Although multimodal large language models (MLLMs) exhibit remarkable reasoning capabilities on complex multimodal understanding tasks, they still suffer from the notorious hallucination issue: generating outputs misaligned with obvious visual or factual evidence. Currently, training-based solutions, like direct preference optimization (DPO), leverage paired preference data to suppress hallucinations. However, they risk sacrificing general reasoning capabilities due to the likelihood displacement. Meanwhile, training-free solutions, like contrastive decoding, achieve this goal by subtracting the estimated hallucination pattern from a distorted input. Yet, these handcrafted perturbations (e.g., add noise to images) may poorly capture authentic hallucination patterns. To avoid these weaknesses of existing methods, and realize robust hallucination mitigation (i.e., maintaining general reasoning performance), we propose a novel framework: Decoupling Contrastive Decoding (DCD). Specifically, DCD decouples the learning of positive and negative samples in preference datasets, and trains separate positive and negative image projections within the MLLM. The negative projection implicitly models real hallucination patterns, which enables vision-aware negative images in the contrastive decoding inference stage. Our DCD alleviates likelihood displacement by avoiding pairwise optimization and generalizes robustly without handcrafted degradation. Extensive ablations across hallucination benchmarks and general reasoning tasks demonstrate the effectiveness of DCD, i.e., it matches DPO's hallucination suppression while preserving general capabilities and outperforms the handcrafted contrastive decoding methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。