arXiv:2608.22516cs.CVcs.CL2026-08中稿 · EMNLP

提出新方法与评测集,让长视频理解更可靠

TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding

论文配图:TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding
图 1 · 摘自论文原文
  • 用多轮证据构建机制逐步聚焦关键帧,直到答案稳定
  • 在严格验证下准确率达50.7%,仅需98.7帧/题,效率远超基线
  • 适合需要高可信推理的长视频任务,如智能监控、内容分析

长视频回答只有在解码的帧覆盖所有依赖事件时才具证据支持。现有评估只关注最终答案正确性或预测证据区间,却很少检验方法在作答前实际解码的帧是否完整。为此,我们构建了包含600个问题的VES-Bench基准,涵盖348段公开长视频,每个问题都要求一组共同必要证据区间,可从三个严格度审计方法解码帧是否全覆盖。我们还提出TRACE——一种无需训练的推理代理:它基于原始视觉片段定位答案,逐轮构建证据包,直至答案随证据增长而稳定,并通过同一片段的回检确认一致性。在相同骨干模型下,TRACE以每题98.7帧的代价实现50.7%的正确率,且每个证据区间至少有两帧被解码;相比均匀采样256帧达40.2%的准确率(仅128帧),提升超10个百分点;仅需0.39倍帧数成本即逼近均匀采样256帧表现(63.5%最高准确率)。同时在Video-MME(86.1)、LVBench(75.6)和LongVideoBench(75.1)上保持竞争力。

原文摘要 · Abstract (English)

A long-video answer is evidence-supported only when the frames decoded from the video cover every event the answer depends on. Existing evaluations score final-answer correctness or predicted evidence intervals, but the frames a method decodes before answering are rarely audited, so correct answers can still rest on incomplete observation. We introduce VES-Bench, a 600-question benchmark of Temporal Ordering and Event Counting items over 348 public long videos. Each item carries a jointly necessary set of evidence intervals, letting us audit at three strictness levels whether a method's decoded frames cover every one of them. We also propose TRACE, a training-free agent that grounds answers in raw visual clips, builds an evidence bundle round by round, and stops only when the answer stabilises as the bundle grows and a final pass over the same clips returns the same answer. Under a same-backbone audit, TRACE answers 50.7% of questions correctly with at least two decoded frames inside every evidence interval, at 98.7 frames per question: over 10 points above uniform decoding at 128 frames (40.2%), and within 2.6 points of uniform decoding at 256 frames at 0.39x its frame cost, while reaching the highest answer accuracy in the audit (63.5%). TRACE also stays competitive on Video-MME (86.1), LVBench (75.6), and LongVideoBench (75.1).

长视频理解证据推理视频问答高效采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。