用专家推理诊断视频生成中的物理错误,发现超九成存在可识别漏洞。
Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning
- 让专家分析生成视频中的物理错误,逐帧标注失效行为。
- 93.5%的视角视频和83.3%的外视角视频存在可察觉物理漏洞。
- 适合研究物理一致性、视频生成安全性的学者与开发者使用。
视频生成模型被越来越多地用于叙事、模拟和具身智能等场景。随着模型发展,一个核心问题浮现:生成视频是否遵循真实世界的物理规律?现有评估主要依赖自动化指标或粗粒度的人类判断(如偏好评分或评分表),虽能衡量感知质量,但难以揭示生成动态违反物理约束的具体情形。本文提出 Physion-Eval,一个大规模基准数据集,包含五种前沿生成模型在第一人称与第三人称视角下生成视频的专家推理记录,共10,990条推理轨迹,覆盖22个细粒度物理类别。每段生成视频均源自对应的真实世界参考视频,展示明确的物理过程,并标注了时间定位的异常点、结构化失败类别及自然语言解释。基于该数据集,我们发现当前生成模型在关键物理场景中存在显著缺陷:93.5%的第一人称视频和83.3%的第三人称视频至少存在一处人类可识别的物理错误。我们希望 Physion-Eval 能成为物理真实性评估的新标准,推动具备物理基础的视频生成技术发展。该基准已公开于 https://huggingface.co/datasets/PhysionLabs/Physion-Eval。
原文摘要 · Abstract (English)
Video generation models are increasingly used as world simulators for storytelling, simulation, and embodied AI. As these models advance, a key question arises: do generated videos obey the physical laws of the real world? Existing evaluations largely rely on automated metrics or coarse human judgments such as preferences or rubric-based checks. While useful for assessing perceptual quality, these methods provide limited insight into when and why generated dynamics violate real-world physical constraints. We introduce Physion-Eval, a large-scale benchmark of expert human reasoning for diagnosing physical realism failures in videos generated by five state-of-the-art models across egocentric and exocentric views, containing 10,990 expert reasoning traces spanning 22 fine-grained physical categories. Each generated video is derived from a corresponding real-world reference video depicting a clear physical process, and annotated with temporally localized glitches, structured failure categories, and natural-language explanations of the violated physical behavior. Using this dataset, we reveal a striking limitation of current video generation models: in physics-critical scenarios, 83.3% of exocentric and 93.5% of egocentric generated videos exhibit at least one human-identifiable physical glitch. We hope Physion-Eval will set a new standard for physical realism evaluation and guide the development of physics-grounded video generation. The benchmark is publicly available at https://huggingface.co/datasets/PhysionLabs/Physion-Eval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。