多智能体协作分析长视频，精准定位并理解关键片段。
LongVideoAgent: Multi-Agent Reasoning with Long Videos
- 用主控智能体协调定位与视觉分析双代理，实现精准时间定位。
- 在长视频问答数据集上超越基线模型，准确率显著提升。
- 适合需要细粒度视频理解的场景，如影视分析与教育评测。
近年来，多模态大模型与工具系统在长视频问答中展现出推理潜力。然而，许多方法仍依赖有损压缩或有限工具集，削弱了时间定位能力并遗漏细微线索。本文提出一种多智能体框架：主控大模型协调定位代理以精确定位问题相关片段，并调度视觉代理提取针对性文本观察。主控代理在步数限制下规划，通过强化学习训练以促进简洁、准确且高效的多代理协作。该设计使主控代理聚焦于相关片段，结合字幕与视觉细节，生成可解释的推理路径。在新提出的长视频问答数据集LongTVQA和LongTVQA+（基于TVQA/TVQA+构建）上，本系统显著优于强基线模型。实验还表明，强化学习进一步增强了代理的推理与规划能力。代码与数据将公开于https://longvideoagent.github.io/。
原文摘要 · Abstract (English)
Recent advances in multimodal LLMs and systems that use tools for long-video QA point to the promise of reasoning over hour-long episodes. However, many methods still compress content into lossy summaries or rely on limited toolsets, weakening temporal grounding and missing fine-grained cues. We propose a multi-agent framework in which a master LLM coordinates a grounding agent to localize question-relevant segments and a vision agent to extract targeted textual observations. The master agent plans with a step limit, and is trained with reinforcement learning to encourage concise, correct, and efficient multi-agent cooperation. This design helps the master agent focus on relevant clips via grounding, complements subtitles with visual detail, and yields interpretable trajectories. On our proposed LongTVQA and LongTVQA+ which are episode-level datasets aggregated from TVQA/TVQA+, our multi-agent system significantly outperforms strong non-agent baselines. Experiments also show reinforcement learning further strengthens reasoning and planning for the trained agent. Code and data will be shared at https://longvideoagent.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。