视频问答先感知后推理,动态决定是否思考,更省力更准确。
Perception Before Reasoning: Dynamic Latent Reasoning for Video Understanding and Question Answering

- 先用视觉隐状态快速定位关键信息,再按需添加推理隐状态
- 在9个数据集上平均准确率提升,单次回答不超过20个词
- 适合追求高效、低延迟的视频理解场景
视频问答需将语言问题与视觉证据对齐,并在必要时跨时间推理。现有方法依赖长篇文字链式思考,但许多问题在定位到相关物体、动作或帧后即可回答。我们提出动态隐式推理(DyLaR),首先用短块感知隐状态(编码查询相关的视觉证据)定位问题,再自适应决定是否添加推理隐状态(在隐空间中对证据进行推理)。DyLaR通过在真实视觉证据上对齐感知隐状态,并将验证后的推理过程蒸馏为推理隐状态,再经强化学习优化何时推理。在九个视频基准测试和四个多模态语言模型主干上,DyLaR在不增加响应长度的情况下提升平均准确率。例如,在 Qwen3-VL-4B 上,准确率从 54.0 提升至 58.2,响应长度从 1,220.7 降至 18.5 个词。消融实验表明,有依据的感知隐状态、基于推理语料监督的推理隐状态和自适应路由均能提升准确率。
原文摘要 · Abstract (English)
Video question answering requires models to ground language queries in visual evidence and, when necessary, reason over that evidence across time. Existing methods typically rely on long textual chain-of-thought rationales, even though many questions can be answered as soon as the relevant object, action, or frame is localized. We propose Dynamic Latent Reasoning (DyLaR), which first grounds a question in a short block of perception latents (continuous hidden states that encode query-relevant visual evidence), and then adaptively decides whether to append reasoning latents (continuous thoughts that reason over this evidence in latent space) before answering. DyLaR learns this behavior by grounding perception latents in verified visual evidence and distilling verified rationales into reasoning latents, followed by reinforcement learning that further refines when to reason. Across nine video benchmarks and four multimodal language model backbones, DyLaR improves average accuracy over same-backbone baselines while generating fewer than 20 tokens per query. On Qwen3-VL-4B, for example, DyLaR improves average accuracy over Qwen3-VL-4B-Thinking from 54.0 to 58.2 while reducing response length from 1,220.7 to 18.5 tokens per query. Ablations further show that grounded perception latents, rationale-supervised reasoning latents, and adaptive routing each improve accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。