arXiv:2601.01804cs.CV2026-01被引 1

让视频大模型更懂时间顺序,提升因果推理能力

V-CORE: Temporally Consistent Video Understanding for Video-LLM

  • 引入单向信息流机制,强制时间顺序理解
  • 在NExT-QA上达61.2%准确率,时序推理提升5.2%
  • 4比特微调+消费级显卡即可训练,适合资源有限场景

近期视频大语言模型虽具备强大多模态推理能力,但在需要一致时间顺序和因果连贯性的视频理解任务中仍受限。许多参数高效模型依赖无约束的双向投影器建模帧间交互,导致后帧影响前帧表示,缺乏对视频推理方向性的显式架构约束。为此,我们提出V-CORE,一种参数高效的框架,通过显式时间顺序约束提升视频理解。其核心包含:(1) 可学习空间聚合(LSA),自适应选择显著空间标记以减少冗余;(2) 因果感知时间投影器(CATP),通过块因果注意力与终端动态摘要标记实现结构化单向信息流。该设计保留帧内空间交互,同时确保时间信息严格有序聚合。采用4比特QLoRA与冻结的LLM主干,V-CORE可在单张消费级显卡上高效训练。实验表明,其在挑战性NExT-QA基准上达到61.2%准确率,在MSVD-QA、MSRVTT-QA和TGIF-QA上保持竞争力,时序与因果推理子任务分别提升3.5%和5.2%,直接验证了显式时间约束的重要性。

原文摘要 · Abstract (English)

Recent Video Large Language Models (Video-LLMs) have shown strong multimodal reasoning capabilities, yet remain challenged by video understanding tasks that require consistent temporal ordering and causal coherence. Many parameter-efficient Video-LLMs rely on unconstrained bidirectional projectors to model inter-frame interactions, which can blur temporal ordering by allowing later frames to influence earlier representations, without explicit architectural mechanisms to respect the directional nature of video reasoning. To address this limitation, we propose V-CORE, a parameter-efficient framework that introduces explicit temporal ordering constraints for video understanding. V-CORE consists of two key components: (1) Learnable Spatial Aggregation (LSA), which adaptively selects salient spatial tokens to reduce redundancy, and (2) a Causality-Aware Temporal Projector (CATP), which enforces structured unidirectional information flow via block-causal attention and a terminal dynamic summary token acting as a causal sink. This design preserves intra-frame spatial interactions while ensuring that temporal information is aggregated in a strictly ordered manner. With 4-bit QLoRA and a frozen LLM backbone, V-CORE can be trained efficiently on a single consumer GPU. Experiments show that V-CORE achieves strong performance on the challenging NExT-QA benchmark, reaching 61.2% accuracy, and remains competitive across MSVD-QA, MSRVTT-QA, and TGIF-QA, with gains concentrated in temporal and causal reasoning subcategories (+3.5% and +5.2% respectively), directly validating the importance of explicit temporal ordering constraints.

视频理解大模型时序推理参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。