用多角色LoRA链实现视频时序精准推理，让回答有视觉证据支撑。
VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning
- 设计四角色代理流程：规划、定位、验证、作答，分工明确。
- 在15个基准上超越现有方法，长视频推理性能显著提升。
- 适合做视频理解、智能客服和教育助手的开发者参考。
视频具有独特的时序维度，要求答案与可解释的视觉证据精确对应。尽管大语言模型在文本推理上取得突破，多模态推理尤其是视频领域仍受限。本文提出VideoMind，一种面向时序锚定视频推理的新颖视频-语言代理。核心创新包括：(1) 识别出四类关键能力，设计基于角色的智能体工作流，包含规划者、定位者、验证者和回答者；(2) 提出Chain-of-LoRA机制，通过统一基模型搭配多个LoRA适配器，在推理中实现角色无缝切换，兼顾效率与灵活性。在15个基准（涵盖Grounded VideoQA、Video Temporal Grounding和General VideoQA）上的实验表明，该方案在视频智能体、测试时扩展和长视频推理方面均具显著优势。代码、模型、数据集及演示已公开于https://videomind.github.io/。
原文摘要 · Abstract (English)
Videos, with their unique temporal dimension, demand precise grounded understanding, where answers are directly linked to visual, interpretable evidence. Despite significant breakthroughs in text-based reasoning with large language models, multi-modal reasoning - especially for videos - remains limited. In this work, we fill this gap by introducing VideoMind, a novel video-language agent for temporal-grounded video reasoning. Our method involves two key innovations: (1) We identify four essential capabilities for grounded video reasoning and propose a role-based agentic workflow, comprising a planner to coordinate roles, a grounder for temporal event localization, a verifier to assess event candidates, and an answerer for question answering. (2) To efficiently integrate these roles during inference, we propose a novel Chain-of-LoRA mechanism, where a unified base model with multiple LoRA adapters is leveraged to enable seamless role switching, balancing efficiency and flexibility. Extensive experiments on 15 benchmarks across Grounded VideoQA, Video Temporal Grounding, and General VideoQA tasks demonstrate the effectiveness of the proposed scheme in advancing video agent, test-time scaling, and long-form video reasoning. Code, models, datasets, and demos are available at https://videomind.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。