让文字生成的3D场景自动符合物理规律，还能直接用于仿真。
PAT3D: Physics-Augmented Text-to-3D Scene Generation
- 用视觉语言模型理解文本，生成物体并构建层次化场景结构。
- 通过可微分刚体模拟器使物体在重力下自然稳定，避免穿插。
- 支持后续编辑和机器人操作，适合需要真实物理的下游任务。
我们提出PAT3D，首个融合物理模拟的文本到3D场景生成框架，将视觉语言模型与基于物理的仿真结合，生成具有物理合理性、可直接用于仿真的无交集3D场景。给定文本提示后，PAT3D生成3D物体，推断其空间关系，并组织为层次化场景树，进而转换为仿真初始条件。一个可微分的刚体模拟器确保物体在重力作用下的真实互动，推动场景达到静态平衡且无穿插。为进一步提升场景质量，引入仿真闭环优化流程，保障物理稳定性与非交叠性，同时增强与输入提示的语义一致性。实验表明，PAT3D在物理合理性、语义一致性和视觉质量上显著优于现有方法。除高质量生成外，PAT3D还首次实现可直接用于下游任务（如场景编辑、机器人操作）的仿真就绪3D场景。代码与数据已开源：https://github.com/Simulation-Intelligence/PAT3D。
原文摘要 · Abstract (English)
We introduce PAT3D, the first physics-augmented text-to-3D scene generation framework that integrates vision-language models with physics-based simulation to produce physically plausible, simulation-ready, and intersection-free 3D scenes. Given a text prompt, PAT3D generates 3D objects, infers their spatial relations, and organizes them into a hierarchical scene tree, which is then converted into initial conditions for simulation. A differentiable rigid-body simulator ensures realistic object interactions under gravity, driving the scene toward static equilibrium without interpenetrations. To further enhance scene quality, we introduce a simulation-in-the-loop optimization procedure that guarantees physical stability and non-intersection, while improving semantic consistency with the input prompt. Experiments demonstrate that PAT3D substantially outperforms prior approaches in physical plausibility, semantic consistency, and visual quality. Beyond high-quality generation, PAT3D uniquely enables simulation-ready 3D scenes for downstream tasks such as scene editing and robotic manipulation. Code and data are available at: https://github.com/Simulation-Intelligence/PAT3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。