arXiv:2608.08622cs.CV2026-08

提出VADER框架,用自适应去偏提升视频大模型幻觉抑制能力。

VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models

论文配图:VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models
图 1 · 摘自论文原文
  • 根据视频内容动态调整视觉注意力分配,实现精准干预。
  • 在LLaVA-Video-7B上事件幻觉任务准确率达72.60%。
  • 适合关注视频理解中幻觉问题的研究者与开发者。

大型视觉语言模型(LVLM)在开放式视频理解任务中表现优异,但常生成与视频证据不符的流畅回答。现有无需训练的方法通常采用全局固定的视觉干预或通过输入扰动生成对比分支,前者难以适配视频依赖的融合路径,后者易被相邻帧冗余补偿。为此,我们提出无需训练的视频自适应去偏框架VADER,包含两个互补模块:视觉焦点重分配(VFR)自动为每个视频-问题输入构建干预策略,诊断层间视觉到文本证据流动,确定干预位置及强度,将预软化注意力从系统词块重分配至视频词块;选择性证据擦除(SEE)独立掩码每帧高重要性视觉标记,构造难以通过邻近帧补偿的先验偏差分支。对比解码机制则降低在选择性证据擦除后仍保持自信的预测权重。在多个VideoLLM上,VADER显著提升事件级定位与时间一致性表现;在LLaVA-Video-7B上,事件幻觉任务准确率达到72.60%。

原文摘要 · Abstract (English)

Large vision-language models (LVLMs) have demonstrated strong performance in open-ended video understanding, yet they remain prone to fluent responses unsupported by video evidence. Existing training-free methods typically apply a globally fixed visual intervention or construct a contrastive branch through input perturbation. The former cannot accommodate video-dependent fusion paths, while the latter can be compensated by cross-frame redundancy. We therefore propose Video-Adaptive Debiasing via Evidence Reweighting (VADER), a training-free framework with two complementary modules. Visual Focus Reallocation (VFR) automatically instantiates an intervention policy for each video-question input: it diagnoses layer-wise visual-to-text evidence flow, determines where to intervene, and derives how strongly to reallocate pre-softmax attention from system-token to video-token blocks. Selective Evidence Erasure (SEE) independently masks high-importance visual tokens in every frame, constructing a prior-biased branch that is difficult to compensate through neighboring frames. Contrastive decoding then down-weights predictions that remain confident after selective evidence erasure. Across multiple VideoLLMs, VADER yields substantial improvements on event-level grounding and temporal consistency; on LLaVA-Video-7B, it reaches 72.60% accuracy on EventHallusion.

视频大模型幻觉抑制自适应干预

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。