测试视频大模型对时间动态的理解能力，发现当前模型严重依赖静态画面。
TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs
- 设计对比视频对，仅改变时间顺序以隔离时序理解难度
- 顶尖模型在时间推理任务上准确率仅48.2%，远低于人类的98.2%
- 适合研究视频理解、具身智能与模型可解释性的学者使用
细粒度时空理解对视频推理和具身智能至关重要。尽管多模态大语言模型（MLLMs）已掌握静态语义，但其对时间动态的理解仍不稳定。我们提出TimeBlind，一个用于组合式时空理解的诊断基准。受认知科学启发，TimeBlind将细粒度时间理解分为三个层次：识别原子事件、刻画事件属性、推理事件间依赖关系。不同于混淆识别与时间推理的基准，TimeBlind采用最小对范式：视频对具有完全相同的静态视觉内容，仅在时间结构上不同，并通过互补问题消除语言先验。在600个精心筛选的实例（共2400个视频-问题对）上评估超过20个前沿MLLM（如GPT-5、Gemini 3 Pro），结果显示最佳模型的实例准确率（正确区分一对视频的能力）仅为48.2%，远低于人类表现（98.2%）。结果表明，即使最先进模型仍严重依赖静态视觉捷径，而非真正的时序逻辑，凸显TimeBlind作为下一代视频理解诊断工具的重要价值。数据集与代码已公开于https://baiqi-li.github.io/timeblind_project/。
原文摘要 · Abstract (English)
Fine-grained spatio-temporal understanding is essential for video reasoning and embodied AI. Yet, while Multimodal Large Language Models (MLLMs) master static semantics, their grasp of temporal dynamics remains brittle. We present TimeBlind, a diagnostic benchmark for compositional spatio-temporal understanding. Inspired by cognitive science, TimeBlind categorizes fine-grained temporal understanding into three levels: recognizing atomic events, characterizing event properties, and reasoning about event interdependencies. Unlike benchmarks that conflate recognition with temporal reasoning, TimeBlind leverages a minimal-pairs paradigm: video pairs share identical static visual content but differ solely in temporal structure, utilizing complementary questions to neutralize language priors. Evaluating over 20 state-of-the-art MLLMs (e.g., GPT-5, Gemini 3 Pro) on 600 curated instances (2400 video-question pairs), reveals that the Instance Accuracy (correctly distinguishing both videos in a pair) of the best performing MLLM is only 48.2%, far below the human performance (98.2%). These results demonstrate that even frontier models rely heavily on static visual shortcuts rather than genuine temporal logic, positioning TimeBlind as a vital diagnostic tool for next-generation video understanding. Dataset and code are available at https://baiqi-li.github.io/timeblind_project/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。