让视频生成更符合物理规律,提升真实感。
Physics-Informed Video Generation via Mixture-of-Experts Latent Alignment

- 用物理属性引导预训练模型的生成过程
- 在多个基准上同时提升画质与物理合理性
- 适合关注真实动态模拟的研究者
大规模视频生成模型在语义一致性和视觉质量方面取得了显著进展,生成的视频越来越连贯且逼真。然而,基于像素级拟合的动力学机制难以自然捕捉现实世界运动与交互的规律,导致物理合理性仍存缺陷。为此,本文提出PILA(Physics-Informed Latent Alignment)框架,将结构化物理潜空间引导注入预训练视频模型的冻结流匹配动力学中。PILA首先通过锚定场估计,将冻结生成器的潜变量映射到按场代理槽组织的物理属性库中,以可观测运动为运动学锚点构建难以直接观测的代理。为应对真实世界动态的异质性,PILA采用物理类别上的专家混合设计;标签先验掩码专家路由选择类别特定的操作专家,其修正由抽象自物理关系的操作残差正则化。最终,优化后的代理被融合进物理属性库,并解码为对流匹配向量场的修正,注入物理感知引导,同时保留预训练主干的视觉先验。在Wan 2.1-1.3B上分阶段适配训练,并直接迁移至Wan 2.2-14B,PILA在VBench-2.0、VideoPhy-2和PhyGenBench三个基准上均达到当前最优,在视觉质量与衡量的物理合理性上均有显著提升。
原文摘要 · Abstract (English)
Large-scale video generation models have made remarkable progress in semantic consistency and visual quality, producing videos that are increasingly coherent and visually convincing. Nevertheless, the dynamics induced by pixel-level fitting do not naturally accommodate the regularities that govern real-world motion and interaction, resulting in persistent shortcomings in physical plausibility. To address this limitation, we propose \textbf{PILA} (Physics-Informed Latent Alignment), a framework that injects physics-structured latent guidance into the frozen flow-matching dynamics of pretrained video models. Specifically, PILA first employs anchored field estimation to map frozen-generator latents into an operational physical attribute bank organized by field-proxy slots, using observable motion as a kinematic anchor for constructing less directly observed proxies. To handle the heterogeneity of real-world dynamics, PILA adopts a mixture-of-experts design over physical categories. Label-prior masked expert routing selects category-specific operator experts, whose refinements are regularized by operational residuals abstracted from physical relations. Finally, the refined proxies are fused into the physical attribute bank and decoded into a correction to the flow-matching vector field, injecting physics-aware guidance while preserving the visual prior of the pretrained backbone. With staged adapter training on Wan 2.1-1.3B and direct transfer of the learned adapter to Wan 2.2-14B, PILA achieves state-of-the-art results on VBench-2.0, VideoPhy-2, and PhyGenBench in both visual quality and benchmark-measured physical plausibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。