发现视频模型在事件计数上存在低频陷阱，难以捕捉高频瞬时事件。
The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping
- 设计可控视频任务，用可执行事件轨迹精准评估模型表现。
- 高频率下仅0.2%计数正确，事件召回率仅18.1%。
- 提升帧率能骗过评分，但无法真实恢复事件序列，适合诊断模型缺陷。
现实世界视频基准测试覆盖广泛，但固定片段将事件数量、频率、持续时间与视觉复杂度纠缠在一起，导致失败模式难以分离。现有程序化基准虽控制更好，却仅评估最终答案，无法审计报告事件与可执行真实事件的匹配度。为此，我们引入基于事件轨迹的参数化评测方法，针对三种受控视频任务：弹球墙击、视觉眨眼和类别状态转换。在2,190个视频中，我们固定渲染条件，系统变化事件数N与频率F。每个视频附带可执行事件轨迹，用于能力曲面估计与时间戳级评估。结果揭示分阶段的时间失效现象：在80%可靠性阈值下，Gemini 3.6 Flash可可靠计数持续状态转换至12个事件（0.5和1.0 Hz），但在瞬时眨眼任务中无任何可靠正计数区域。事件表示方式决定模型能否初始获取证据，该限制随数量与频率上升而加剧。在高计数、高频场景下，仅有0.2%的最终计数正确，模型仅恢复18.1%的真实事件。为检验视觉感知是否为主要瓶颈，我们提高采样率，虽使弹球任务准确率从19.6%升至29.3%，但报告序列与真实轨迹一致率仅3.7%。额外帧可虚增得分，却无法实现忠实事件恢复。不同提示策略带来相似局限性，真实世界视频评估也呈现成功集中于低事件数的现象。最终，基于轨迹的评测将视频评估从整体准确率转向对时间推理失效位置的精细诊断。
原文摘要 · Abstract (English)
Real-world video benchmarks provide broad coverage, but their fixed clips entangle event count, rate, duration, and visual complexity, making failure modes hard to isolate. While existing programmatic benchmarks offer better control, they score only the final answer rather than auditing reported events against executable ground truth. To bridge this gap, we introduce trace-grounded parametric profiling for event counting in three controlled video tasks: bouncing-ball wall contacts, visual blinks, and categorical state transitions. Across 2,190 videos, we vary event count N and frequency F while holding rendering fixed. Each video includes an executable event trace for capability-surface estimation and timestamp-level evaluation. Our results reveal a staged temporal failure. At an 80% reliability threshold, Gemini 3.6 Flash reliably counts persistent state transitions up to 12 events at 0.5 and 1.0 Hz, yet demonstrates no reliable positive-count region for transient blinking events. Thus, event representation dictates whether a model initially accesses evidence -- a limitation that compounds as count and frequency increase. In the high-count, high-frequency regime, only 0.2% of final counts are correct and the model recovers just 18.1% of true events. To test if visual access is the primary bottleneck, we increase sampling rate. Although this boosts Bounce Ball accuracy from 19.6% to 29.3%, the reported sequence agrees with ground truth only 3.7% of the time. Extra frames can therefore inflate final scores without producing faithful event recovery. Different prompting strategies yield similarly limited gains, and real-world video evaluations show the same concentration of success at low event counts. Ultimately, trace-grounded profiling shifts video evaluation from aggregate accuracy metrics to a detailed diagnostic of where temporal reasoning fails.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。