让流式世界模型学会真实物理规律,提升长期预测的合理性。
Distilling Physical Priors into Streaming World Models

- 用12万条真实物理交互视频训练,注入物体属性与因果变化知识
- 将大模型物理先验蒸馏到小模型,实现高效实时生成
- 通过在线强化学习优化物理一致性,适合长序列视频生成任务
流式世界模型在长时间范围内预测未来视觉状态时,常违反基本物理规律。现有方法将预训练的双向DiT蒸馏为几步自回归生成器,但存在两个根本问题:通用双向教师模型从视觉预训练中获取的物理先验有限,且在双向到因果蒸馏过程中进一步损失。本文提出PhyS,一种三阶段框架,将物理先验注入流式世界模型。首先构建包含12万条真实物理交互视频的数据集PhyS-120K,涵盖刚体动力学、软体形变、流体现象和相变;每段视频附带物体属性与因果状态转移的结构化标注。通过物理感知监督微调,将物理先验注入140亿参数的双向DiT教师模型,再蒸馏为13亿参数的轻量级因果DiT,实现少步自回归流式生成。最后,采用在线强化学习激励模型生成物理合理轨迹,并提出时间信用路由(TCR)解决时间信用分配问题:在重叠时间窗口上评估物理一致性,将组内相对优势分配给时间对齐的去噪动作。在PhysicsIQ上,PhyS相比Wan2.1-14B教师提升18.2%,相比Self Forcing、Rolling Forcing和Causal Forcing分别提升23.7%、14.8%、31.4%。在VideoPhy、VideoPhy2和PhyGenBench等物理感知视频基准上也取得改进。数据集、代码及更多样例视频见项目主页。
原文摘要 · Abstract (English)
Streaming world models predict future visual states online while maintaining physically coherent dynamics over long horizons. However, their rollouts often violate basic physical constraints. A common approach distills pretrained bidirectional DiTs into few-step causal generators. However, this paradigm suffers from two fundamental limitations: generic bidirectional teachers acquire limited physical priors from visually oriented pretraining, and the limited priors suffer further loss during bidirectional-to-causal distillation. We present PhyS, a three-stage framework for distilling physical priors into streaming world models. To acquire physical priors from real-world interactions, we construct PhyS-120K, a dataset of 120K real-world physical-interaction videos spanning rigid-body dynamics, soft-body deformation, fluid phenomena, and phase transitions. Each video is annotated with structured descriptions of object properties and causal state transitions. Physics-aware supervised fine-tuning injects the physical priors into a bidirectional 14B DiT teacher, which we then distill into a lightweight 1.3B causal DiT for few-step autoregressive streaming generation. Finally, we use online reinforcement learning to incentivize the distilled model to generate physically plausible rollouts and further propose Temporal Credit Routing (TCR) to address temporal credit assignment. TCR evaluates physical consistency over overlapping temporal windows and routes the resulting group-relative advantages to temporally aligned denoising actions. On PhysicsIQ, PhyS improves the Wan2.1-14B teacher by 18.2\% and the Self Forcing, Rolling Forcing, and Causal Forcing by 23.7\%, 14.8\%, and 31.4\%, respectively. Results also improve the physics-aware video benchmarks VideoPhy, VideoPhy2, and PhyGenBench. The dataset, code, and more sample videos are available on our Project Page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。