用单图生成符合物理规律的动态3D内容,速度快且可控。
Phy124: Fast Physics-Driven 4D Content Generation from a Single Image

- 将物理仿真直接融入4D生成流程,确保动作真实
- 去除扩散模型,生成速度提升显著,推理时间大幅缩短
- 通过外力控制运动速度和方向,适合交互式应用
4D内容生成旨在创建随时间变化的动态3D物体。现有方法主要依赖预训练视频扩散模型,通过采样过程或参考视频生成,但存在两大挑战:其一,生成内容常违背真实物理规律,因扩散模型未融入物理先验;其二,采样过程复杂且模型参数量大,导致生成耗时极长。为此,本文提出Phy124,一种新颖、快速且基于物理驱动的单图可控4D内容生成方法。Phy124将物理仿真直接嵌入4D生成过程,确保生成内容遵循自然物理法则。同时,在4D动态生成阶段摒弃扩散模型,极大提升速度。用户可通过调节外部力控制造型的运动速度与方向。大量实验表明,Phy124生成的4D内容保真度高,推理时间显著降低,达到当前最优性能。代码与生成结果已公开于 https://anonymous.4open.science/r/BBF2/。
原文摘要 · Abstract (English)
4D content generation focuses on creating dynamic 3D objects that change over time. Existing methods primarily rely on pre-trained video diffusion models, utilizing sampling processes or reference videos. However, these approaches face significant challenges. Firstly, the generated 4D content often fails to adhere to real-world physics since video diffusion models do not incorporate physical priors. Secondly, the extensive sampling process and the large number of parameters in diffusion models result in exceedingly time-consuming generation processes. To address these issues, we introduce Phy124, a novel, fast, and physics-driven method for controllable 4D content generation from a single image. Phy124 integrates physical simulation directly into the 4D generation process, ensuring that the resulting 4D content adheres to natural physical laws. Phy124 also eliminates the use of diffusion models during the 4D dynamics generation phase, significantly speeding up the process. Phy124 allows for the control of 4D dynamics, including movement speed and direction, by manipulating external forces. Extensive experiments demonstrate that Phy124 generates high-fidelity 4D content with significantly reduced inference times, achieving stateof-the-art performance. The code and generated 4D content are available at the provided link: https://anonymous.4open.science/r/BBF2/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。