发现视频模型常靠常识猜顺序，提出新方法提升时序理解能力。
What Happens When: Learning Temporal Orders of Events in Videos
- 用事件描述+思维链提示训练模型，增强时序感知。
- 在新基准上，多数模型时序判断准确率不足50%。
- 适合关注视频理解时序逻辑的研究者与开发者。
视频多模态模型在视频理解任务中表现优异，但对多个事件时间顺序的准确捕捉能力仍不充分。我们发现，即使视频帧被随机打乱，现有模型在基准测试中仍表现良好，表明这些模型可能并非依赖视觉事件的精确时序处理，而是依赖典型场景的先验知识作答。为此，我们提出了VECTOR基准，专门评估模型识别事件时序的能力。实验显示，多种视频多模态模型在该基准上常失败。为此，我们提出MECOT（基于思维链的多事件指令微调），通过详细事件级描述训练，并在推理时引入思维链提示以增强时序意识。MECOT在VECTOR上优于已有方法，同时提升了现有基准的表现，验证了其有效性。代码、模型和数据集已公开。
原文摘要 · Abstract (English)
Video Large Multimodal Models (VLMMs) have shown impressive performance in video understanding, yet their ability to accurately capture the temporal order of multiple events remains underexplored. We interestingly observe that, even when video frames are scrambled, models perform very well on the existing benchmarks by comprehensive experiments. This implies that VLMMs may not necessarily rely on accurate sequential processing of visual events, but instead depend on prior knowledge of typical scenarios to answer the question. To benchmark temporal understanding capabilities in VLMMs, we propose VECTOR, designed to explicitly assess a model's ability to identify the temporal order of events. On this benchmark, we observe that various VLMMs often fail to understand the orders of events. To address this, we propose MECOT (Multi-Event instruction fine-tuning with Chain-of-Thought), which (1) trains models on detailed, event-by-event video descriptions and (2) using chain-of-thought prompts at inference to enhance temporal awareness. MECOT outperforms prior arts on VECTOR as well as improving performance on existing video benchmarks, implying effectiveness of temporal understanding. We release our code, model and datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。