MUPA通过多路径协同推理，让视频问答更可信。
MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering
- 设计三种不同顺序的视觉与问答代理协作路径
- 在NExT-GQA和DeVE-QA上分别达到30.3%和47.4%准确率
- 仅用20亿参数就超越多数70亿参数模型
基于视频的问答任务需要将文本答案与明确的视觉证据对齐。然而，现代多模态模型常依赖语言先验和虚假关联，导致预测缺乏可靠依据。本文提出MUPA，一种合作式多路径智能体推理框架，统一了视频定位、问题回答、答案反思与结果聚合。MUPA在不同时间顺序下设计三种推理路径，并引入专门的反思代理来评估和融合多路径结果，实现一致的问答与定位。该设计显著提升了定位可靠性，且不牺牲回答准确率。尽管仅使用20亿参数，其表现已超越所有70亿参数的竞争模型。当扩展至70亿参数时，MUPA在NExT-GQA和DeVE-QA上分别取得30.3%和47.4%的Acc@GQA，展现出在可信赖视频-语言理解中的有效性。代码已开源。
原文摘要 · Abstract (English)
Grounded Video Question Answering (Grounded VideoQA) requires aligning textual answers with explicit visual evidence. However, modern multimodal models often rely on linguistic priors and spurious correlations, resulting in poorly grounded predictions. In this work, we propose MUPA, a cooperative MUlti-Path Agentic approach that unifies video grounding, question answering, answer reflection and aggregation to tackle Grounded VideoQA. MUPA features three distinct reasoning paths on the interplay of grounding and QA agents in different chronological orders, along with a dedicated reflection agent to judge and aggregate the multi-path results to accomplish consistent QA and grounding. This design markedly improves grounding fidelity without sacrificing answer accuracy. Despite using only 2B parameters, our method outperforms all 7B-scale competitors. When scaled to 7B parameters, MUPA establishes new state-of-the-art results, with Acc@GQA of 30.3% and 47.4% on NExT-GQA and DeVE-QA respectively, demonstrating MUPA' effectiveness towards trustworthy video-language understanding. Our code is available in https://github.com/longmalongma/MUPA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。