让多个智能体协作找视频证据，比单次扫描更准
AgentCVR: Active Multi-Agent Cross-Video Reasoning via Script-Simulated Reinforcement Learning
- 用主控智能体调度视觉/音频专精智能体，分步主动找证据
- 在基准测试中超越单次处理模型，复杂对齐任务表现优秀
- 用文本脚本模拟训练，省去昂贵多模态推理，适合复现
跨视频推理（CVR）是多模态智能的关键挑战，要求模型从多段视频中检索、对齐并聚合分散的证据。当前多模态大模型常因单次编码将多视频压缩到共享上下文，可能掩盖罕见但关键的信息。本文提出AgentCVR，一种将CVR视为主动证据获取任务的多智能体框架。该框架由主控智能体协调视觉与音频专用智能体，迭代执行目标化证据提取。为高效训练，引入脚本模拟强化学习（Script-Simulated RL），利用大语言模型生成语义脚本和轻量级文本模拟器，避免在线探索时进行昂贵的多模态推理。在综合性CVR基准上的实验表明，AgentCVR优于单次处理基线，并在复杂跨视频对齐与定位任务中达到与先进闭源系统相当的性能。代码已开源：https://github.com/wang-jh24/AgentCVR。
原文摘要 · Abstract (English)
Cross-Video Reasoning (CVR) has emerged as a critical frontier in multimodal intelligence, requiring models to retrieve, align, and aggregate evidence distributed across multiple videos. Current Multimodal Large Language Models (MLLMs) often struggle with CVR, as simple single-pass strategies encode multiple videos into a shared compressed context, potentially obscuring rare but critical evidence. In this paper, we propose AgentCVR, a multi-agent framework that treats CVR as an active evidence-acquisition task. AgentCVR employs a Master Agent to iteratively coordinate specialized Visual and Audio Agents for targeted evidence extraction. To ensure efficient training, we introduce Script-Simulated RL, which optimizes the agent's policy with LLM-generated semantic scripts and a lightweight text-based simulator, bypassing costly multimodal inference during online exploration. Experimental results on a comprehensive CVR benchmark show that AgentCVR outperforms single-pass baselines and achieves comparable performance to state-of-the-art closed-source systems, particularly in complex cross-video alignment and localization. To ensure reproducibility, our code is available at https://github.com/wang-jh24/AgentCVR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。