arXiv:2608.04587cs.CV2026-08被引 1

自动演化视频理解代理,提升长视频问答准确率。

MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding

论文配图:MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding
图 1 · 摘自论文原文
  • 基于采样帧与问题分析信息密度,动态设计初始视频代理。
  • 通过最小化验证任务定位失败模块,实现精准迭代优化。
  • 在8个数据集上迭代4次,准确率提升13.03个百分点。

长视频理解需在多模态、长序列视频中定位稀疏的问答相关证据。真实视频分布存在模态信息密度、内容结构和证据模式差异,导致固定设计的视频代理常产生冗余计算或失效。将自动化代理演化从文本扩展至视频面临挑战:完整长视频执行代价高,错误在耦合处理阶段传播,且复杂预处理、感知工具与定位策略使代码更新难以可靠实施。本文提出MetaVideoAgent框架,可自动为特定视频分布演化视频代理。该框架通过稀疏采样帧与关联问题分析信息密度与证据需求,指导初始设计;将局部失败压缩为独立可执行的最小验证任务;构建基于证据的黄金路径,审计学生轨迹,聚合跨样本重复失败并归因于责任模块。采用模块化代理表示,确保每次更新仅作用于主责模块及其必要依赖。进一步提出VA-EvoBench基准,覆盖8个视频分布,含独立演化与保留测试集。每分布4次演化迭代后,所有初始代理均被改进,宏平均准确率从38.44%提升至51.47%,平均每分布演化成本3.54M token。演化代理超越最强先验固定设计代理6.39个百分点,且在单位问题消耗的令牌数与视频帧数上最少。代码与数据将开源以支持可复现研究。

原文摘要 · Abstract (English)

Long-form video understanding requires locating sparse, question-relevant evidence in long, multimodal videos. Real-world video distributions differ in modality-specific information density, content structure, and evidence patterns, causing fixed video-agent designs to incur redundant processing or fail when mismatched. Extending automated agent evolution from text to video is challenging because full long-video execution makes candidate validation expensive, failures propagate across coupled evidence-processing stages, and complex preprocessing, perception tools, and localization strategies make code-level updates difficult to implement reliably. We introduce MetaVideoAgent, a framework that automatically evolves a video agent for a target distribution. It profiles information density and evidence requirements from sparsely sampled frames and associated queries to guide initial design, then compresses localized failures into independently executable minimal validation tasks. It constructs evidence-grounded Gold Paths, audits Student trajectories, aggregates recurring failures across samples, and attributes them to responsible modules. A modular agent representation constrains each update to the primary responsible module and its necessary dependencies. We further introduce VA-EvoBench, covering eight video distributions with separate evolution and held-out splits. With four evolution iterations per distribution, MetaVideoAgent improves every initial agent and raises macro-average accuracy from 38.44% to 51.47%, at an average evolution cost of 3.54M tokens per distribution. The evolved agents outperform the strongest prior fixed-design video agent by 6.39 percentage points while using the fewest tokens and video frames per question among the compared video agents. We will release all code and data to support reproducible research.

视频理解自动演化代理系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。