提升视频模型物理理解评估的可靠性，优化基准测试方法。
Physics-IQ Verified
- 优化提示与真实视频质量，减少干扰因素影响
- 引入样本级评分系统，均衡各指标权重
- 适用于评估视频生成模型的物理真实性
视频生成模型（VGMs）已不仅用于视频生成，还可支持世界建模等下游任务。要推进这些任务，模型必须理解物理现实。为此，物理理解评估成为新方向，催生了Physics-IQ基准，通过对比模型生成视频与真实物理实验视频来量化物理理解能力。本文对Physics-IQ基准进行系统性审计，揭示其缺陷并提出三项改进：提升提示与真实视频质量以减少混淆因素，引入样本级评分系统实现各样本与指标的均等加权。改进后的Physics-IQ Verified基准修正了57.6%的样本，优化了超过34.8%的提示。在六种图像到视频生成模型的对比研究中，观察到中等但显著的排名变化（Kendall's $τ=0.46$）。我们希望Physics-IQ Verified为社区提供更可靠的物理准确性信号。代码可在https://github.com/google-deepmind/physics-iq-benchmark获取。
原文摘要 · Abstract (English)
Video generative models ( VGMs) have become a new frontier that can be used not just for video generation but for a multitude of downstream tasks, including world modeling. To advance these tasks, a good video model must understand the physical reality of the world. Evaluating this understanding is an emerging field and has led to the Physics-IQ benchmark, which quantifies this explicitly by comparing model-generated videos to real-world videos of physical experiments. In this work, we present a systematic audit of the Physics-IQ benchmark, expose shortcomings and propose three solutions that sharpen how we can measure physical understanding of VGMs. Specifically, we improve prompt and ground-truth quality to reduce the influence of confounding factors and further introduce a sample-level scoring system that weights each sample and metric equally. Our resulting benchmark, Physics-IQ Verified, refines 57.6\% of all samples and improves over 34.8\% of prompts. In a comparison study using six image-to-video generative models, we observe moderate but meaningful ranking changes (Kendall's $τ= 0.46$). We hope Physics-IQ Verified advances the community by providing a more reliable signal toward physically accurate VGMs. The code for the benchmark can be accessed at https://github.com/google-deepmind/physics-iq-benchmark
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。