通过不确定性检测大模型幻觉，无需标注数据。
VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation
- 用语义等价的扰动提示测试模型响应一致性。
- 响应越不一致，说明模型越不确定，越可能幻觉。
- 适用于评估视觉语言模型安全性的研究人员。
相比单模态大语言模型，大型视觉语言模型（LVLM）处理的信息量更大，导致其幻觉检测需更多人工与时间成本，引发更广泛的安全担忧。本文提出首个基于不确定性的幻觉检测框架VL-Uncertainty。不同于依赖真实或伪标注的现有方法，该框架利用不确定性作为内在指标：通过分析语义等价但经过扰动的视觉与文本提示下的预测方差来度量不确定性。当模型高度自信时，对语义等价查询会给出一致响应；当不确定时，响应则趋于随机。针对语义相似但表述不同的回答，我们基于语义内容聚类并计算聚类分布熵作为不确定性度量，以识别幻觉。在4个基准上对10个LVLM进行的广泛实验表明，该方法在自由问答和多选任务中均显著优于强基线方法。
原文摘要 · Abstract (English)
Given the higher information load processed by large vision-language models (LVLMs) compared to single-modal LLMs, detecting LVLM hallucinations requires more human and time expense, and thus rise a wider safety concerns. In this paper, we introduce VL-Uncertainty, the first uncertainty-based framework for detecting hallucinations in LVLMs. Different from most existing methods that require ground-truth or pseudo annotations, VL-Uncertainty utilizes uncertainty as an intrinsic metric. We measure uncertainty by analyzing the prediction variance across semantically equivalent but perturbed prompts, including visual and textual data. When LVLMs are highly confident, they provide consistent responses to semantically equivalent queries. However, when uncertain, the responses of the target LVLM become more random. Considering semantically similar answers with different wordings, we cluster LVLM responses based on their semantic content and then calculate the cluster distribution entropy as the uncertainty measure to detect hallucination. Our extensive experiments on 10 LVLMs across four benchmarks, covering both free-form and multi-choice tasks, show that VL-Uncertainty significantly outperforms strong baseline methods in hallucination detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。