分离规划与答案权威,提升长视频问答的准确性和可解释性。
VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authority

- 将推理过程拆分为规划与验证两阶段,通过像素级验证确保答案有据可依。
- 在LVBench和LongVideoBench上分别达到55.1%和62.0%准确率,显著提升证据对齐度。
- 框架可扩展、支持模型替换,无需重训规划模块,适合实际部署。
长视频问答需在高度冗余的内容中定位稀疏且时间分散的视觉证据。尽管当前多模态大模型在短视频上表现良好,长视频带来的长时程搜索与验证仍需多轮代理交互。我们发现现有长视频理解代理存在“证据错位”:给出正确答案却无对应证据支持。为此,提出两个诊断指标(时间锚定性与语义锚定性),揭示两类加剧错位的压力——推理时共享上下文饱和引发的提示压力,以及训练中仅基于结果优化带来的奖励压力。根源在于代理架构将长期规划与答案权威耦合。为此提出解耦的规划-检查器框架,分离规划与答案生成,并以像素级验证作为最终回答前提。在四个长视频基准上,该框架同时提升准确率与证据对齐度,在LVBench上达55.1%,LongVideoBench上达62.0%,并生成可解释的搜索轨迹。该结构可随搜索预算增加稳定扩展,支持无须重训的MLLM主干替换。代码与模型已公开于https://github.com/Echochef/VideoSEAL。
原文摘要 · Abstract (English)
Long video question answering requires locating sparse, time-scattered visual evidence within highly redundant content. Although current MLLMs perform well on short videos, long videos introduce long-horizon search and verification, which often necessitates multi-turn, agentic interaction. We show that existing LVU agents can exhibit "evidence misalignment": they produce correct answers that are not supported by the retrieved or inspected evidence. To characterize this failure, we introduce two diagnostics (temporal groundedness and semantic groundedness) and use them to reveal two pressures that amplify misalignment: prompt pressure from shared-context saturation at inference time and reward pressure from outcome-only optimization during training. These findings point to a structural root cause: the coupled agent paradigm conflates long-horizon planning with answer authority. We therefore propose the decoupled planner-inspector framework, which separates planning from answer authority and gates final answering on pixel-level verification. Across four long-video benchmarks, our framework improves both answer accuracy and evidence alignment, achieving 55.1% on LVBench and 62.0% on LongVideoBench while producing interpretable search trajectories. Moreover, the decoupled architecture scales consistently with increased search budgets and supports plug-and-play upgrades of the MLLM backbone without retraining the planner. Code and models are available at https://github.com/Echochef/VideoSEAL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。