单图生成带物理属性的4D动态模型,速度快且真实感强。
PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesis
- 从单张图像联合预测3D高斯点云与物理属性,无需逐场景优化
- 1分钟内完成高质量4D模拟,速度远超现有方法
- 自建5万+物理标注资产数据集,支持真实物理行为学习
尽管基于物理的3D运动合成取得进展,现有方法仍存在三大局限:依赖耗时的多视角图像重建3D高斯点云(3DGS)并需逐场景优化;物理融合方式要么僵化手动设定,要么依赖高成本视频模型引导的评分蒸馏采样(SDS);以及简单拼接预构建3DGS与物理模块,忽略外观中嵌入的物理信息,导致性能不佳。为此,我们提出PhysGM,一种前馈式框架,能从单张图像联合预测3D高斯表示与物理属性,实现即时模拟与高保真4D渲染。不同于缓慢的外观无关优化方法,我们首先预训练一个具备物理感知能力的重建模型,直接推断高斯与物理参数。进一步通过直接偏好优化(DPO)微调模型,使其模拟结果与物理合理参考视频对齐,避免高成本的SDS优化。为解决该任务缺乏数据集的问题,我们构建了包含5万+3D资产、附带物理属性和参考视频的PhysAssets数据集。实验表明,PhysGM仅用一分钟即可从单图生成高保真4D模拟,在速度上显著超越先前工作,同时呈现逼真视觉效果。
原文摘要 · Abstract (English)
Despite advances in physics-based 3D motion synthesis, current methods face key limitations: reliance on pre-reconstructed 3D Gaussian Splatting (3DGS) built from dense multi-view images with time-consuming per-scene optimization; physics integration via either inflexible, hand-specified attributes or unstable, optimization-heavy guidance from video models using Score Distillation Sampling (SDS); and naive concatenation of prebuilt 3DGS with physics modules, which ignores physical information embedded in appearance and yields suboptimal performance. To address these issues, we propose PhysGM, a feed-forward framework that jointly predicts 3D Gaussian representation and physical properties from a single image, enabling immediate simulation and high-fidelity 4D rendering. Unlike slow appearance-agnostic optimization methods, we first pre-train a physics-aware reconstruction model that directly infers both Gaussian and physical parameters. We further refine the model with Direct Preference Optimization (DPO), aligning simulations with the physically plausible reference videos and avoiding the high-cost SDS optimization. To address the absence of a supporting dataset for this task, we propose PhysAssets, a dataset of 50K+ 3D assets annotated with physical properties and corresponding reference videos. Experiments show that PhysGM produces high-fidelity 4D simulations from a single image in one minute, achieving a significant speedup over prior work while delivering realistic renderings. Our project page is at:https://hihixiaolv.github.io/PhysGM.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。