让视频问答模型自动识别并纠正虚构引用的错误。
Catching Hallucinated Citations in Video-LLM Question Answering: A Self-Verification Pipeline and Verifier Ablation Study
- 用检索增强模型生成带时间戳的回答，再逐帧验证引用真实性。
- 新验证器可捕获79%的虚假引用，且不误伤真实答案。
- 适合关注视频问答可信度的研究者与开发者使用。
基于视觉-语言模型的视频问答系统常在无依据的情况下生成带时间戳的断言，且以高置信度呈现，造成误导性幻觉。为解决此问题，本文提出一个闭环验证流程：先由检索增强语言模型生成带时间戳的答案，再对每个引用帧独立进行复核。实验对比了三种验证设计，在Apple Silicon（MLX）和Google Colab（HF Transformers, CUDA）上评估。直接询问视觉模型是否支持断言的策略失败（40个断言中0%捕获率），因模型存在迎合倾向；盲重描述+通用大模型判断虽提升但结果不稳定（0%至100%波动）；改用小型自然语言推理模型作为判断器后，实现稳定、可解释的验证，可在对抗性假前提问题上捕获79%的伪造引用，同时保持真实引用不变。代码已开源，支持多平台部署。
原文摘要 · Abstract (English)
Video question answering systems built on vision-language models often produce timestamped claims with high confidence even when unsupported by the cited frame. This deceptive hallucination arises because timestamps imply grounding without ensuring correctness, increasing user trust but not accuracy. We introduce a pipeline that closes this loop. A retrieval-augmented language model drafts answers with per-claim timestamp citations, and each cited frame is independently re-examined before being shown to the user. We compare against a plain baseline and ablate three verification designs, evaluated on both Apple Silicon (MLX) and Google Colab (HF Transformers, CUDA). Directly asking the vision model whether a frame supports a claim fails completely (0% catch rate on 40 claims) due to sycophancy. Blind re-captioning plus a general LLM judge improves results but is unstable, oscillating between 0% and 100% flagged depending on prompt phrasing. Replacing that judge with a small natural language inference model yields a stable, interpretable verifier that catches 79% of fabricated claims on adversarial false-premise questions while leaving true claims untouched. We release the full pipeline, evaluation harness, and implementations for both Apple Silicon and Colab. Code is available at https://github.com/yogesh-iitj/grounded-video-qa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。