给AI视频生成加个时间脉搏,让动作速度更真实
The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics

- 从视频视觉动态直接推断真实帧率(PhyFPS)
- 发现主流生成模型存在严重时间尺度错乱问题
- 适合关注视频时序真实性与物理合理性的研究者
尽管近期生成视频模型在视觉真实感上取得显著进展,并被探索用于世界建模,但真正的物理模拟需要同时掌握空间与时间。现有模型虽能生成视觉流畅的运动,却缺乏可靠的内部运动脉冲来锚定其运动在一致的真实时间尺度上。这种时间模糊性源于对不同实际速度视频的无差别训练,迫使模型统一到标准帧率,导致我们称之为‘时间幻觉’的现象:生成序列的运动速度模糊、不稳定且不可控。为此,我们提出视觉计时器(Visual Chronometer),通过受控的时间重采样训练,直接从输入视频的视觉动态中恢复物理帧率(PhyFPS)。为系统量化该问题,我们建立了两个基准:PhyFPS-Bench-Real 和 PhyFPS-Bench-Gen。评估显示,顶尖视频生成模型存在严重的 PhyFPS 错位和时间不稳定性。最后,我们证明应用 PhyFPS 修正可显著提升生成视频的人类感知自然度。
原文摘要 · Abstract (English)
While recent generative video models have achieved remarkable visual realism and are being explored as world models, true physical simulation requires mastering both space and time. Current models can produce visually smooth kinematics, yet they lack a reliable internal motion pulse to ground these motions in a consistent, real-world time scale. This temporal ambiguity stems from the common practice of indiscriminately training on videos with vastly different real-world speeds, forcing them into standardized frame rates. This leads to what we term chronometric hallucination: generated sequences exhibit ambiguous, unstable, and uncontrollable physical motion speeds. To address this, we propose Visual Chronometer, a predictor that recovers the Physical Frames Per Second (PhyFPS) directly from the visual dynamics of an input video. Trained via controlled temporal resampling, our method estimates the true temporal scale implied by the motion itself, bypassing unreliable metadata. To systematically quantify this issue, we establish two benchmarks, PhyFPS-Bench-Real and PhyFPS-Bench-Gen. Our evaluations reveal a harsh reality: state-of-the-art video generators suffer from severe PhyFPS misalignment and temporal instability. Finally, we demonstrate that applying PhyFPS corrections significantly improves the human-perceived naturalness of AI-generated videos. Our project page is https://xiangbogaobarry.github.io/Visual_Chronometer/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。