用单图或文本生成可动3D物体,保持结构与运动一致
ArtGen: Conditional Generative Modeling of Articulated Objects in Arbitrary Part-Level States
- 基于扩散模型,通过跨状态采样确保运动结构一致
- 在PartNet-Mobility上生成质量优于现有方法
- 适合机器人、数字孪生等领域需精准可动资产的场景
生成可动三维物体对机器人、数字孪生和具身智能至关重要。现有生成模型多依赖闭合状态的单视角输入,导致几何与关节动态纠缠,产生模糊或不合理的运动结构。为此,我们提出ArtGen,一种基于扩散模型的条件生成框架,可从单视角图像或文本描述中生成任意部件状态下的可动3D物体,兼具准确几何与连贯运动。ArtGen采用跨状态蒙特卡洛采样,显式强化全局运动一致性,降低结构-运动纠缠;集成思维链推理模块,推断部件语义、关节类型与连接关系等结构先验,指导稀疏专家扩散变换器专注于多样化运动交互;同时,结合局部-全局注意力的组合式3D-VAE潜空间先验,有效捕捉精细几何与整体部件关系。在PartNet-Mobility基准上的大量实验表明,ArtGen显著优于当前最优方法。
原文摘要 · Abstract (English)
Generating articulated assets is crucial for robotics, digital twins, and embodied intelligence. Existing generative models often rely on single-view inputs representing closed states, resulting in ambiguous or unrealistic kinematic structures due to the entanglement between geometric shape and joint dynamics. To address these challenges, we introduce ArtGen, a conditional diffusion-based framework capable of generating articulated 3D objects with accurate geometry and coherent kinematics from single-view images or text descriptions at arbitrary part-level states. Specifically, ArtGen employs cross-state Monte Carlo sampling to explicitly enforce global kinematic consistency, reducing structural-motion entanglement. Additionally, we integrate a Chain-of-Thought reasoning module to infer robust structural priors, such as part semantics, joint types, and connectivity, guiding a sparse-expert Diffusion Transformer to specialize in diverse kinematic interactions. Furthermore, a compositional 3D-VAE latent prior enhanced with local-global attention effectively captures fine-grained geometry and global part-level relationships. Extensive experiments on the PartNet-Mobility benchmark demonstrate that ArtGen significantly outperforms state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。