发现多轮对话中幻觉会不断放大,提出新方法有效遏制。
MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialogue

- 通过双机制刷新视觉表征与输出分布,抑制错误传播
- 在多轮对话中使模型幻觉率降低42%,保持视觉一致性
- 适合关注多模态交互可靠性的研究者与开发者
多模态大语言模型虽具出色视觉理解能力,但在交互场景中易受幻觉雪球效应影响:初始错误随对话轮次递增放大,导致语义崩溃。这暴露出模型逐渐忽视视觉依据、过度依赖污染文本历史的根本缺陷。现有基准多限于单轮视觉问答,无法捕捉长程交互中的误差传播机制。为此,我们提出首个细粒度诊断幻觉雪球效应的基准MM-Snowball。大规模评估表明,该基准对先进多模态模型构成严峻挑战,且现有单轮缓解方法无效。为此,我们提出无需训练的冲突感知视觉修正(CAVR)方法,通过表示层刷新视觉锚点与逻辑层校正输出分布,实现双重抑制。实验显示,CAVR显著优于现有方法,在多轮对话中将幻觉率降低42%,为构建更可靠的交互式AI提供新路径。数据与代码已公开。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) demonstrate remarkable visual understanding, yet their reliability in interactive settings is severely undermined by hallucination snowballing: a phenomenon where initial errors amplify across conversational turns, leading to a collapse in coherence. This failure reveals a fundamental vulnerability where models progressively neglect visual grounding in favor of over-relying on polluted textual history. Existing benchmarks are predominantly confined to single-turn VQA, which fail to capture the complex dynamics of error propagation in long-horizon interactions. To address this, we introduce MM-Snowball, the first benchmark for fine-grained diagnosis of hallucination snowballing within dialogues. Extensive evaluation shows that our benchmark poses a significant challenge even to advanced MLLMs and reveals the inefficacy of existing mitigation methods designed for single-turn VQA. To counteract this degradation, we propose Conflict-Aware Visual Rectification (CAVR). This training-free method mitigates snowballing through a synergistic dual-mechanism that refreshes visual grounding at the representation level and rectifies output distributions at the logit level, effectively re-anchoring the model to visual facts. Experiments demonstrate that CAVR achieves state-of-the-art performance, offering a promising path toward more reliable interactive AI. Data and code are available at: https://frenkie-chiang.github.io/MM-Snowball
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。