让AI像侦探一样逐帧推理视频,准确找出关键证据
Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence
- 构建多阶段推理框架,自动识别视频中的关键帧和证据
- 在6个基准测试中平均准确率提升超10%,达当前最优
- 适合需要精细视频理解的场景,如安防监控、医疗分析
视频推理需跨帧多步推断,仍是多模态大模型的重大挑战。尽管基于强化学习的方法提升了推理能力,但常依赖纯文本链,导致结论脱离视觉事实或出现幻觉。而帧检索方法虽引入视觉依据,却仍存在证据定位不准的问题。为此,我们提出Conan框架,实现基于证据的多步视频推理。Conan可识别上下文与证据帧,分析跨帧线索,并自适应决定是否终止或继续探索。我们首先构建了Conan-91K数据集,包含自动标注的推理轨迹,涵盖帧识别、证据推理与动作决策;其次设计多阶段渐进式冷启动策略,结合识别-推理-行动(AIR)强化学习视觉推理训练框架,逐步激励多步视觉推理。在六个多步推理基准上的实验表明,Conan相比基线Qwen2.5-VL-7B-Instruct平均准确率提升超过10%,达到当前最佳性能。此外,Conan在长视频理解任务上也展现出良好泛化能力,验证其强可扩展性与鲁棒性。
原文摘要 · Abstract (English)
Video reasoning, which requires multi-step deduction across frames, remains a major challenge for multimodal large language models (MLLMs). While reinforcement learning (RL)-based methods enhance reasoning capabilities, they often rely on text-only chains that yield ungrounded or hallucinated conclusions. Conversely, frame-retrieval approaches introduce visual grounding, yet still struggle with inaccurate evidence localization. To address these limitations, we present Conan, a framework for evidence-grounded multi-step video reasoning. Conan identifies context and evidence frames, reasons over cross-frame clues, and adaptively decides when to conclude or explore further. To achieve this, we 1) construct Conan-91K, a large-scale dataset of automatically generated reasoning traces that include frame identification, evidence reasoning, and action decision, and 2) design a multi-stage progressive cold-start strategy combined with an Identification-Reasoning-Action (AIR) RLVR training framework to progressively incentivize multi-step visual reasoning. Extensive experiments on six multi-step reasoning benchmarks demonstrate that Conan surpasses the baseline Qwen2.5-VL-7B-Instruct by an average of over 10% in accuracy, achieving state-of-the-art performance. Furthermore, Conan generalizes effectively to long video understanding tasks, validating its strong scalability and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。