arXiv:2604.23145cs.CVcs.AI2026-04

用显式推理模块提升视频问答的可解释性与准确率

UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks

论文配图:UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
图 1 · 摘自论文原文
  • 先用多模态大模型生成物体识别和场景上下文,再交由下游模型作答
  • 在OpenEQA和NExTQA上性能提升,但基线高时可能反而下降
  • 适合需要透明决策过程的视频理解任务,如医疗或自动驾驶

视频问答(VideoQA)要求模型联合处理空间、时间与语言线索。然而,该任务常需多步推理,当前大型多模态模型(LMMs)通常隐式完成,决策过程不透明。相比之下,大型推理模型(LRMs)会显式生成中间逻辑步骤,提升可解释性并改善多跳推理准确率。但这些模型未针对原生视频理解设计,通常依赖静态帧采样。我们提出UpstreamQA,一个模块化框架,通过显式上游推理模块解耦并评估核心视频推理组件。具体而言,使用多模态LRM(o4-mini, Gemini 2.5 Pro)进行物体识别与场景上下文生成,再将增强的推理轨迹传递给下游LMM(GPT-4o, Gemini 2.5 Flash)完成VideoQA。在OpenEQA和NExTQA数据集上的实验表明,引入显式推理可显著提升下游VideoQA的性能与可解释性,但在基线性能足够高时可能导致性能下降。总体而言,UpstreamQA提供了一个结合显式推理与多模态理解的合理框架,在多个场景中推进了VideoQA的性能与诊断透明性。

原文摘要 · Abstract (English)

Video Question Answering (VideoQA) demands models that jointly reason over spatial, temporal, and linguistic cues. However, the task's inherent complexity often requires multi-step reasoning that current large multimodal models (LMMs) perform implicitly, leaving their internal decision process opaque. In contrast, large reasoning models (LRMs) explicitly generate intermediate logical steps that enhance interpretability and can improve multi-hop reasoning accuracy. Yet, these models are not designed for native video understanding, as they typically rely on static frame sampling. We propose UpstreamQA, a modular framework that disentangles and evaluates core video reasoning components through explicit upstream reasoning modules. Specifically, we employ multimodal LRMs to perform object identification and scene context generation before passing enriched reasoning traces to downstream LMMs for VideoQA. We evaluate UpstreamQA on the OpenEQA and NExTQA datasets using two LRMs (o4-mini, Gemini 2.5 Pro) and two LMMs (GPT-4o, Gemini 2.5 Flash). Our results demonstrate that introducing explicit reasoning can significantly boost performance and interpretability of downstream VideoQA, but can also lead to performance degradation when baseline performance is sufficiently high. Overall, UpstreamQA offers a principled framework for combining explicit reasoning and multimodal understanding, advancing both performance and diagnostic transparency in VideoQA in several scenarios.

视频问答显式推理可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。