通过稳定度感知解码,减少视觉语言模型幻觉
SAKED: Mitigating Hallucination in Large Vision-Language Models via Stability-Aware Knowledge Enhanced Decoding
- 基于内部知识稳定性设计解码机制,识别三类幻觉模式
- 在多个模型和任务上显著降低幻觉率,性能领先
- 无需训练,适配多种架构,适合部署在真实场景
大型视觉语言模型(LVLM)的幻觉问题在实际应用中带来安全与可靠性风险。受人类在不确定时更易出错的启发,我们研究了模型内部知识不稳定性对幻觉的影响。从注意力头、模型层和解码词元三个角度开展实证分析,发现三类关键幻觉模式:(i) 注意力头间视觉激活漂移,(ii) 层间知识剧烈波动,(iii) 相邻输出词元间的视觉焦点干扰。基于此,提出稳定性感知知识增强解码(SAKED),引入层级知识稳定度评分(KSS)量化知识稳定性。通过对比最稳定与最不稳定的层,抑制解码噪声,动态利用最可靠的内部知识生成准确词元。SAKED无需训练,可无缝集成到不同架构中。大量实验表明,其在多种模型、任务和基准上均实现领先的幻觉缓解效果。
原文摘要 · Abstract (English)
Hallucinations in Large Vision-Language Models (LVLMs) pose significant security and reliability risks in real-world applications. Inspired by the observation that humans are more error-prone when uncertain or hesitant, we investigate how instability in a model 's internal knowledge contributes to LVLM hallucinations. We conduct extensive empirical analyses from three perspectives, namely attention heads, model layers, and decoding tokens, and identify three key hallucination patterns: (i) visual activation drift across attention heads, (ii) pronounced knowledge fluctuations across layers, and (iii) visual focus distraction between neighboring output tokens. Building on these findings, we propose Stability-Aware Knowledge-Enhanced Decoding (SAKED), which introduces a layer-wise Knowledge Stability Score (KSS) to quantify knowledge stability throughout the model. By contrasting the most stability-aware and stability-agnostic layers, SAKED suppresses decoding noise and dynamically leverages the most reliable internal knowledge for faithful token generation. Moreover, SAKED is training-free and can be seamlessly integrated into different architectures. Extensive experiments demonstrate that SAKED achieves state-of-the-art performance for hallucination mitigation on various models, tasks, and benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。