arXiv:2603.14052cs.CVcs.MA2026-03中稿 · CVPR

多智能体协作解析长视频,效率与准确率双提升。

A Multi-Agent Perception-Action Alliance for Efficient Long Video Reasoning

  • 用多轮感知-行动循环,分步提取关键线索并定位事件片段。
  • 在5个基准上超越18种模型,推理延迟显著降低。
  • 适合需要高效处理长视频的AI研究者与开发者。

本文提出一种名为A4VL的多智能体感知-行动探索联盟,用于高效长视频推理。A4VL采用多轮感知-行动探索循环,由多个视觉语言模型(VLM)智能体协同工作。每轮中,智能体先通过感知探索从少量采样帧中提取与查询相关的感知线索,并基于线索对齐找到最相关的视频片段;再通过行动探索进行三步推理:(1)各智能体生成初步答案;(2)相互交叉评分与相关性排序;(3)根据是否达成满意共识,决定是否进入下一轮感知-行动迭代(如剔除表现最差智能体并重新采样线索和匹配块),或输出最终答案。该机制结合事件驱动的片段划分与线索引导的块对齐,使A4VL能有效扩展至真实世界长视频,同时保持高质量推理。在五个主流VideoQA基准上的评估显示,A4VL优于18种现有VLM及11种专为长视频优化的方法,且推理延迟显著更低。代码已开源:https://github.com/git-disl/A4VL。

原文摘要 · Abstract (English)

This paper presents a multi-agent perception-action exploration alliance, dubbed A4VL, for efficient long-video reasoning. A4VL operates in a multi-round perception-action exploration loop with a selection of VLM agents. In each round, the team of agents performs video question-answer (VideoQA) via perception exploration followed by action exploration. During perception exploration, each agent learns to extract query-specific perception clue(s) from a few sampled frames and performs clue-based alignment to find the video block(s) that are most relevant to the query-specific event. During action exploration, A4VL performs video reasoning in three steps: (1) each agent produces its initial answer with rational, (2) all agents collaboratively scores one another through cross-reviews and relevance ranking, and (3) based on whether a satisfactory consensus is reached, the decision is made either to start a new round of perception-action deliberation by pruning (e.g., filtering out the lowest performing agent) and re-staging (e.g., new-clue and matching block based perception-action exploration), or to conclude by producing its final answer. The integration of the multi-agent alliance through multi-round perception-action exploration, coupled with event-driven partitioning and cue-guided block alignment, enables A4VL to effectively scale to real world long videos while preserving high quality video reasoning. Evaluation Results on five popular VideoQA benchmarks show that A4VL outperforms 18 existing representative VLMs and 11 recent methods optimized for long-video reasoning, while achieving significantly lower inference latency. Our code is released at https://github.com/git-disl/A4VL.

视频理解多智能体长视频推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。