统一生成刚体、柔体和关节物体的物理仿真3D资产,提升生成与理解能力。
PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects

- 设计新几何表示,直接编码高分辨率3D结构,无需压缩提升生成效果。
- 构建首个通用仿真3D数据集PhysXVerse,覆盖室内外多种类别。
- 提出PhysX-Bench评测基准,全面评估生成与理解能力,适合机器人等应用。
仿真可用的物理3D资产因其在下游任务中的广泛应用前景而受到关注。然而,现有3D生成方法大多忽略物理属性,或仅限于单一资产类型,如刚体、柔体或关节物体。为此,我们提出PhysX-Omni,一个统一的跨多种资产类型的仿真可用物理3D生成框架。具体而言,我们开发了一种专为视觉-语言模型设计的新颖高效几何表示,直接编码高分辨率3D结构,无需压缩,显著提升生成性能。同时,我们构建了首个通用仿真可用3D数据集PhysXVerse,涵盖多样化的室内与室外类别。此外,为全面且灵活地评估生成与理解能力,我们提出了PhysX-Bench评测基准,包含六个关键属性:几何、绝对尺度、材质、可操作性、运动学和功能描述。大量实验表明,PhysX-Omni在生成与理解方面均表现优异。进一步研究验证了其在仿真场景生成与机器人策略学习中的潜力。我们认为,PhysX-Omni可显著推动包括具身AI和物理仿真在内的多种下游应用。
原文摘要 · Abstract (English)
Simulation-ready physical 3D assets have emerged as a promising direction owing to their broad applicability in downstream tasks. However, most existing 3D generation methods either neglect physical properties or are limited to a single asset category, e.g., rigid, deformable, or articulated objects. To address these limitations, we introduce PhysX-Omni, a unified framework for simulation-ready physical 3D generation across diverse asset types. Specifically, we develop a novel and efficient geometry representation tailored for Vision-Language Models, which directly encodes high-resolution 3D structures without compression, significantly improving generation performance. In addition, we construct the first general simulation-ready 3D dataset, PhysXVerse, covering diverse indoor and outdoor categories. Furthermore, to comprehensively and flexibly evaluate both generative and understanding capabilities in the wild, we propose PhysX-Bench, which encompasses six key attributes: geometry, absolute scale, material, affordance, kinematics, and function description. Extensive experiments with conventional metrics and PhysX-Bench show that PhysX-Omni performs strongly in both generation and understanding. Moreover, additional studies further validate the potential of PhysX-Omni for applications in simulation-ready scene generation and robotic policy learning. We believe PhysX-Omni can significantly advance a wide range of downstream applications, particularly in embodied AI and physics-based simulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。