arXiv:2507.12465cs.CV2025-07NeurIPS被引 41

让3D模型具备真实物理属性,提升仿真与机器人应用可行性

PhysX-3D: Physical-Grounded 3D Asset Generation

论文配图:PhysX-3D: Physical-Grounded 3D Asset Generation
图 1 · 摘自论文原文
  • 构建首个系统标注五大物理维度的3D数据集PhysXNet
  • 提出PhysXGen框架,实现图像生成带物理属性的3D资产
  • 适合做物理仿真、具身智能和生成式AI的研究者

3D建模正从虚拟走向物理世界。现有3D生成模型多关注几何与纹理,忽视物理属性,导致合成资产难以用于仿真与具身智能等实际场景。为此,我们提出端到端的物理驱动3D资产生成框架PhysX-3D:1)构建首个系统标注绝对尺度、材料、可操作性、运动学和功能描述五维物理属性的3D数据集PhysXNet,采用基于视觉语言模型的人机协同标注流程,高效生成物理优先的3D资产;2)提出PhysXGen,一种前馈式图像到3D资产的生成框架,将物理知识注入预训练3D结构空间。该框架采用双分支结构,显式建模3D结构与物理属性间的潜在关联,生成兼具合理物理预测与原生几何质量的3D资产。大量实验证明其性能优越且泛化能力强。代码、数据与模型将全部开源,推动生成式物理人工智能研究。

原文摘要 · Abstract (English)

3D modeling is moving from virtual to physical. Existing 3D generation primarily emphasizes geometries and textures while neglecting physical-grounded modeling. Consequently, despite the rapid development of 3D generative models, the synthesized 3D assets often overlook rich and important physical properties, hampering their real-world application in physical domains like simulation and embodied AI. As an initial attempt to address this challenge, we propose \textbf{PhysX-3D}, an end-to-end paradigm for physical-grounded 3D asset generation. 1) To bridge the critical gap in physics-annotated 3D datasets, we present PhysXNet - the first physics-grounded 3D dataset systematically annotated across five foundational dimensions: absolute scale, material, affordance, kinematics, and function description. In particular, we devise a scalable human-in-the-loop annotation pipeline based on vision-language models, which enables efficient creation of physics-first assets from raw 3D assets.2) Furthermore, we propose \textbf{PhysXGen}, a feed-forward framework for physics-grounded image-to-3D asset generation, injecting physical knowledge into the pre-trained 3D structural space. Specifically, PhysXGen employs a dual-branch architecture to explicitly model the latent correlations between 3D structures and physical properties, thereby producing 3D assets with plausible physical predictions while preserving the native geometry quality. Extensive experiments validate the superior performance and promising generalization capability of our framework. All the code, data, and models will be released to facilitate future research in generative physical AI.

3D生成物理模拟具身智能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。