利用图像编码器的几何特征，快速判断视频是否符合物理规律。
GEOPHYS: The Geometry of Physical Plausibility
- 从冻结图像编码器的帧嵌入中提取五种几何特性作为物理合理性指标。
- 在两个数据集上达到98.3%和93.3%的检测准确率，远超现有模型。
- 可高效用于视频生成验证，显著提升生成质量且节省资源。
人类能在毫秒级识别物理上不合理的事件，而现有机器学习方法要么速度慢、成本高，要么依赖外部多模态大模型或需修改训练流程。本文提出，冻结图像编码器生成的每帧嵌入中隐含了五种几何特性，统称为GEOPHYS，能有效反映物理合理性。实验表明，这些特性与人类对物体恒常性违背的脑电反应相关；在LikePhys和IntPhys2数据集上，其物理违规检测准确率分别达98.3%和93.3%，显著优于V-JEPA 2、GPT-4o、Gemini及12个现代视频扩散模型（均接近随机水平）。将其作为生成过程中的最佳候选验证器，可使MAGI-1 24B在PhysicsIQ测试中得分从50.01%提升至64.50%，同时减少1.5倍运行时间和4.65倍内存消耗。结果证明，通过分析图像编码器的时序特征几何结构，可高效评估视频的物理合理性。
原文摘要 · Abstract (English)
While humans can identify physically implausible events within milliseconds, machine learning approaches addressing the same problem are extremely slow and expensive. They either rely on external multimodal-LLM judges or require ad-hoc modifications to the training procedure. In this work, we argue that indicators of physical plausibility are implicitly captured by five geometric properties of the per-frame embeddings produced by frozen image encoders. In aggregate, we call them GEOPHYS. First, we show that these signals correlate with human EEG responses to two forms of object-permanence violations. Second, GEOPHYS robustly discriminates physically implausible videos from realistic ones, achieving state-of-the-art physics-violation detection: 98.3% on LikePhys and 93.3% on IntPhys2, whereas V-JEPA 2, GPT-4o, Gemini, and twelve modern video diffusion models perform near chance. Third, used as a best-of-N verifier for physical alignment during video generation, GEOPHYS lifts MAGI-1 24B from 50.01% to 64.50% on PhysicsIQ at 1.5x lower wall-clock and 4.65x lower memory than the V-JEPA 2 world-model verifier. Ultimately, GEOPHYS demonstrates that physical plausibility in videos can be assessed by leveraging the emergent geometric properties of temporal features extracted from image encoders.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。