用扩散模型补全被遮挡的3D部件,让物体分割更完整。
HoloPart: Generative 3D Part Amodal Segmentation
- 先用现有方法获取不完整的部件分割,再用扩散模型补全
- 在ABO和PartObjaverse-Tiny数据集上显著优于现有方法
- 适合需要精确3D几何编辑与动画的应用场景
3D部件无遮挡分割——将3D形状分解为完整且语义明确的部件,即使存在遮挡——是3D内容创作与理解中的关键挑战。现有3D部件分割方法仅识别可见表面区域,限制了实际应用。受2D无遮挡分割启发,我们首次将该任务引入3D领域,并提出一种两阶段实用方法,解决推断被遮挡3D几何、保持整体形状一致性及在有限训练数据下处理多样化形状等核心难题。首先,利用现有3D部件分割获得初始不完整的部件片段;其次,提出HoloPart,一种基于扩散模型的新方法,完成这些片段为完整3D部件。HoloPart采用专用架构,结合局部注意力捕捉精细部件几何,全局形状上下文注意力确保整体一致性。我们在ABO和PartObjaverse-Tiny数据集上构建新基准,结果表明HoloPart显著优于当前最优的形状补全方法。结合现有分割技术,实现有前景的3D部件无遮挡分割效果,为几何编辑、动画与材质分配开辟新路径。
原文摘要 · Abstract (English)
3D part amodal segmentation--decomposing a 3D shape into complete, semantically meaningful parts, even when occluded--is a challenging but crucial task for 3D content creation and understanding. Existing 3D part segmentation methods only identify visible surface patches, limiting their utility. Inspired by 2D amodal segmentation, we introduce this novel task to the 3D domain and propose a practical, two-stage approach, addressing the key challenges of inferring occluded 3D geometry, maintaining global shape consistency, and handling diverse shapes with limited training data. First, we leverage existing 3D part segmentation to obtain initial, incomplete part segments. Second, we introduce HoloPart, a novel diffusion-based model, to complete these segments into full 3D parts. HoloPart utilizes a specialized architecture with local attention to capture fine-grained part geometry and global shape context attention to ensure overall shape consistency. We introduce new benchmarks based on the ABO and PartObjaverse-Tiny datasets and demonstrate that HoloPart significantly outperforms state-of-the-art shape completion methods. By incorporating HoloPart with existing segmentation techniques, we achieve promising results on 3D part amodal segmentation, opening new avenues for applications in geometry editing, animation, and material assignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。