让大模型像人一样分步找视频重点，提升长视频理解能力。
Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding
- 构建分层时间树，按关键帧递进定位视频片段。
- 在相同帧数预算下，三项评测平均提升4.5%以上。
- 无需训练，适合需精准定位的长视频分析场景。
长视频理解对多模态大模型仍具挑战，受限于上下文长度。均匀采样可能遗漏关键时刻，而基于智能体的帧级理解方法常独立评估帧，忽视视频的时间结构。理想情况下，证据筛选应模拟人类回答长视频问题的过程：先从全局中定位相关片段，再聚焦局部对象与细节。我们提出 T^3（Temporal Tree of Thought），一种无需训练的自适应粗粒度到细粒度长视频理解框架。T^3 通过递归时间约束聚类构建与问题无关的分层时间树，每个节点代表一个连续片段及其信息性关键帧。推理时，T^3 执行“推理-检索-探索”循环：在粗粒度代表性帧上推理，若证据不足则生成搜索语句，并扩展相关分支以获取更细粒度证据。该过程自适应地将搜索目标从时间区域转向具体对象和视觉细节，助力视频理解。在 VideoMME、LongVideoBench 与 LVBench 上的实验表明，T^3 在相同帧预算下使 Qwen2.5-VL-7B 的性能分别提升 0.5%、4.6% 和 4.4%，验证了结构化时间推理的有效性。
原文摘要 · Abstract (English)
Long-video understanding remains challenging for Multimodal Large Language Models (MLLMs) due to limited context length. Uniform sampling may miss crucial moments, while agent-based frame video understanding methods often evaluate frames independently, overlooking the temporal organization of videos. Ideally, evidence selection should mimic how humans answer questions about long videos: first locating the relevant segment from the global context, then zooming into local objects and details. We propose Temporal Tree of Thought T^3, a training-free framework for adaptive coarse-to-fine long-video understanding. T^3 constructs a question-agnostic hierarchical temporal tree via recursive temporally constrained clustering, where each node represents a contiguous segment with an informative key frame. During inference, T^3 performs an answer-retrieve-explore loop: it reasons over coarse representative frames, generates a search statement when evidence is insufficient, and expands relevant branches for finer-grained evidence. This process adaptively shifts the search target from temporal regions to specific objects and visual details to help video understanding. Experiments on VideoMME, LongVideoBench, and LVBench show that T^3 improves Qwen2.5-VL-7B by 0.5%, 4.6%, and 4.4%, respectively, under the same frame budget, demonstrating the effectiveness of structured temporal reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。