用证据理论量化视觉语言模型的内部不确定性,精准识别幻觉与异常输出。
Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification
- 将模型输出特征解析为支持或反对证据,用证据理论建模冲突与无知。
- 在四种错误类型上均优于基线,幻觉对应高冲突,分布外输入对应高无知。
- 单次前向传播完成检测,适合部署于安全敏感的AI应用中。
大型视觉语言模型(LVLMs)在多模态理解与生成方面取得显著进展,但在面对能力不足或对抗性输入时,常产生不可靠甚至有害的内容,如事实幻觉或危险指令。这种与人类预期不符的行为称为LVLM的误操作,源于认知不确定性,具体表现为内部知识冲突或缺乏支持信息。现有不确定性量化方法仅捕捉整体认知不确定性,难以有效识别此类问题。为此,本文提出证据不确定性量化(EUQ),一种细粒度方法,能同时捕捉信息冲突与知识缺失。我们把模型输出头的特征解释为正(支持)或负(反对)证据,利用证据理论建模并聚合这些证据,在一次前向传播中量化内部冲突与知识缺口。我们在四个误操作类别(幻觉、越狱攻击、对抗脆弱性、分布外失败)上对前沿LVLMs进行了广泛评估,结果表明EUQ始终优于强基线:幻觉对应高内部冲突,分布外失败对应高无知。此外,逐层分析揭示了内部表征演化的全新视角。源代码已开源:https://github.com/HT86159/EUQ。
原文摘要 · Abstract (English)
%Large vision-language models (LVLMs) have shown substantial advances in multimodal understanding and generation. However, when presented with incompetent or adversarial inputs, they frequently produce unreliable or even harmful content, such as fact hallucinations or dangerous instructions. This misalignment with human expectations, referred to as \emph{misbehaviors} of LVLMs, raises serious concerns for deployment in critical applications. These misbehaviors are found to stem from epistemic uncertainty, specifically either conflicting internal knowledge or the absence of supporting information. However, existing uncertainty quantification methods, which typically capture only overall epistemic uncertainty, have shown limited effectiveness in identifying such issues. To address this gap, we propose Evidential Uncertainty Quantification (EUQ), a fine-grained method that captures both information conflict and ignorance for effective detection of LVLM misbehaviors. In particular, we interpret features from the model output head as either supporting (positive) or opposing (negative) evidence. Leveraging Evidence Theory, we model and aggregate this evidence to quantify internal conflict and knowledge gaps within a single forward pass. %We extensively evaluate our method across four categories of misbehavior, including hallucinations, jailbreaks, adversarial vulnerabilities, and out-of-distribution (OOD) failures, using state-of-the-art LVLMs, and find that EUQ consistently outperforms strong baselines, showing that hallucinations correspond to high internal conflict and OOD failures to high ignorance. Furthermore, layer-wise evidential uncertainty dynamics analysis helps interpret the evolution of internal representations from a new perspective. The source code is available at https://github.com/HT86159/EUQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。