通过结构化证据定位提升多视频事件理解的准确性和引用完整性。
TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation

- 先用OCR和目标检测构建可搜索的时间线,再用文本模型精确定位关键片段。
- 在MAGMaR 2026上将MiRAGE F1从0.705提升至0.811,引用召回率从0.440升至0.628。
- 适合需要高精度事实追溯与跨视频引用的新闻分析、赛事回放等场景。
多视频事件理解要求模型能在长而异构的视频语料中定位并关联查询相关的证据。现有大视觉语言模型(LVLM)在此场景下表现不佳,因其迅速耗尽上下文预算,难以精确定位具有证据意义的片段,常遗漏广播字幕、标题、比分板等密集信息线索。我们提出TRACE,一种基于证据定位引导的框架,采用‘先定位后推理’策略进行多视频事件推理。方法首先利用OCR和目标检测为每段视频构建结构化、可文本检索的时间线;随后由纯文本大模型执行查询感知的证据定位,选出相关时刻后再进行下游视觉推理。所提取帧及其定位摘要用于驱动基于LVLM的声明生成与跨视频引证整合。在MAGMaR 2026和WikiVideo上的实验表明,结构化定位显著提升事实完整性和引用准确性。在MAGMaR验证集上,TRACE将宏平均MiRAGE F1从0.705提升至0.811,引用召回率从0.440增至0.628。该方法还在官方MAGMaR 2026排行榜上取得最优性能。代码已开源:https://github.com/pengyu965/TRACE。
原文摘要 · Abstract (English)
Multi-video event understanding demands models that can locate and attribute query-relevant evidence scattered across long, heterogeneous video corpora. Existing large vision-language models (LVLMs) often underperform in this regime because they quickly exhaust their context budget and struggle to precisely localize evidentially important segments, frequently missing dense informational cues such as broadcast graphics, subtitles, and scoreboards. We introduce TRACE, an evidence grounding-guided framework that follows a ground-before-reasoning strategy for multi-video event reasoning. Our approach first builds a structured, text-searchable timeline for each video using OCR and object detection. A text-only LLM then conducts query-aware evidence localization, selecting relevant moments prior to any downstream visual reasoning. The retrieved frames and their grounding summaries are subsequently used to steer LVLM-based claim generation and cross-video citation consolidation. Experiments on MAGMaR 2026 and WikiVideo demonstrate that structured grounding markedly boosts factual completeness and attribution fidelity. On the MAGMaR validation split, TRACE raises macro-average MiRAGE F1 from 0.705 to 0.811 compared to an unguided Qwen3-VL-30B baseline, with especially strong improvements in citation recall from 0.440 to 0.628. The method also attains state-of-the-art results on the official MAGMaR 2026 leaderboard. Code is released at https://github.com/pengyu965/TRACE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。