arXiv:2505.11741cs.AIcs.CR2025-05被引 3

通过分析前十个词的可靠性,提升视觉语言模型幻觉检测准确率

MTRE: Multi-Token Reliability Estimation for Hallucination Detection in VLMs

  • 利用前十个词的对数似然比与自注意力机制融合信号
  • 在多个基准上比传统方法准确率提升9.4%,AUROC提升14.8%
  • 适合关注大模型安全性和生成可信度的研究者

视觉语言模型(VLMs)在多项多模态任务中已接近人类表现,但仍存在生成虚假物体或不安全文本的问题。现有幻觉检测方法如单令牌线性探针(LP)和PTrue,通常仅分析首个生成词的输出概率,忽略了早期词序列中的丰富信息。我们发现,幻觉可能在多个词之后才显现,因细微不一致逐步累积。通过分析幻觉与非幻觉词对应对数间的KL散度,我们强调需引入后期词的对数以更准确捕捉模型可靠性动态。为此,提出轻量级白盒方法MTRE,通过多令牌对数似然比与自注意力聚合前十个词的输出。尽管词汇表庞大、对数序列长,MTRE仍保持高效。在MAD-Bench、MM-SafetyBench、MathVista及四个组合几何基准上,MTRE相比标准方法准确率提升9.4%,AUROC提升14.8%,成为开源VLM幻觉检测新基准。

原文摘要 · Abstract (English)

Vision-language models (VLMs) now rival human performance on many multimodal tasks, yet they still hallucinate objects or generate unsafe text. Current hallucination detectors, e.g., single-token linear probing (LP) and PTrue, typically analyze only the logit of the first generated token or just its highest-scoring component, overlooking richer signals embedded within earlier token distributions. We demonstrate that analyzing the complete sequence of early logits potentially provides substantially more diagnostic information. We emphasize that hallucinations may only emerge after several tokens, as subtle inconsistencies accumulate over time. By analyzing the Kullback-Leibler (KL) divergence between logits corresponding to hallucinated and non-hallucinated tokens, we underscore the importance of incorporating later-token logits to more accurately capture the reliability dynamics of VLMs. In response, we introduce Multi-Token Reliability Estimation (MTRE), a lightweight, white-box method that aggregates logits from the first ten tokens using multi-token log-likelihood ratios and self-attention. Despite the challenges posed by large vocabulary sizes and long logit sequences, MTRE remains efficient and tractable. Across MAD-Bench, MM-SafetyBench, MathVista, and four compositional-geometry benchmarks, MTRE achieves a 9.4% gain in accuracy and a 14.8% gain in AUROC over standard detection methods, establishing a new state of the art in hallucination detection for open-source VLMs.

幻觉检测视觉语言模型可靠性估计多令牌

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。