用多个专业智能体协同理解视频，提升问答准确率。
VideoMultiAgents: A Multi-Agent Framework for Video Question Answering
- 分设视觉、场景图、文本处理三类智能体，独立推理增强多模态理解。
- 问题引导生成相关性强的视频描述，使答案准确率提升6.2%。
- 适合需要精准视频理解与复杂推理的应用场景。
视频问答（VQA）依赖多模态推理，需融合视觉、时间与语言信息以深入理解视频内容。然而，现有方法多将帧级描述输入单一模型，难以充分捕捉时间与交互上下文。为此，我们提出VideoMultiAgents框架，集成视觉、场景图分析与文本处理三类专用智能体，通过独立运行的智能体实现互补的多模态推理。同时引入问题引导的描述生成机制，自动生成突出目标物体、动作与时间变化的描述，直接服务于当前问题，显著提升回答准确性。实验表明，该方法在Intent-QA上达到79.0%（较前序SOTA提升6.2%），EgoSchema子集达75.4%（+3.4%），NExT-QA达79.6%（+0.4%）。源代码已开源：https://github.com/PanasonicConnect/VideoMultiAgents。
原文摘要 · Abstract (English)
Video Question Answering (VQA) inherently relies on multimodal reasoning, integrating visual, temporal, and linguistic cues to achieve a deeper understanding of video content. However, many existing methods rely on feeding frame-level captions into a single model, making it difficult to adequately capture temporal and interactive contexts. To address this limitation, we introduce VideoMultiAgents, a framework that integrates specialized agents for vision, scene graph analysis, and text processing. It enhances video understanding leveraging complementary multimodal reasoning from independently operating agents. Our approach is also supplemented with a question-guided caption generation, which produces captions that highlight objects, actions, and temporal transitions directly relevant to a given query, thus improving the answer accuracy. Experimental results demonstrate that our method achieves state-of-the-art performance on Intent-QA (79.0%, +6.2% over previous SOTA), EgoSchema subset (75.4%, +3.4%), and NExT-QA (79.6%, +0.4%). The source code is available at https://github.com/PanasonicConnect/VideoMultiAgents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。