无需文本提示,一键分割3D物体任意部件,支持多粒度灵活识别。
SAMPart3D: Segment Any Part in 3D Objects

- 用视觉模型替代文本提示,实现无标签数据下的大规模3D特征学习。
- 通过多尺度特征提取,可识别不同精细程度的3D部件,精度显著提升。
- 适用于复杂物体分割,适合交互式编辑与3D生成等应用。
3D部件分割是3D感知中的关键挑战任务,在机器人、3D生成和编辑中至关重要。现有方法依赖文本提示进行2D到3D知识迁移,实现零样本分割,但受限于提示依赖,难以扩展至大规模无标签数据集,且处理部件模糊性能力有限。本文提出SAMPart3D,一种可扩展的零样本3D部件分割框架,无需预设标签集即可对任意3D物体按多粒度分割为语义部件。为提升可扩展性,采用不依赖文本的视觉基础模型,从大规模无标签3D数据(如Objaverse)中蒸馏3D特征提取主干,学习丰富3D先验。为增强灵活性,蒸馏出尺度感知的部件敏感3D特征,支持多粒度分割。分割后,利用视觉语言模型(VLMs)基于多视角渲染结果为各部件分配语义标签。实验表明,SAMPart3D显著优于现有零样本方法,能有效处理复杂非标准物体,并推动部件级编辑与交互分割等应用。此外,我们构建了一个新基准以弥补现有数据集在对象多样性与部件复杂性上的不足。
原文摘要 · Abstract (English)
3D part segmentation is a crucial and challenging task in 3D perception, playing a vital role in applications such as robotics, 3D generation, and 3D editing. Recent methods harness the powerful Vision Language Models (VLMs) for 2D-to-3D knowledge distillation, achieving zero-shot 3D part segmentation. However, these methods are limited by their reliance on text prompts, which restricts the scalability to large-scale unlabeled datasets and the flexibility in handling part ambiguities. In this work, we introduce SAMPart3D, a scalable zero-shot 3D part segmentation framework that segments any 3D object into semantic parts at multiple granularities, without requiring predefined part label sets as text prompts. For scalability, we use text-agnostic vision foundation models to distill a 3D feature extraction backbone, allowing scaling to large unlabeled 3D datasets to learn rich 3D priors. For flexibility, we distill scale-conditioned part-aware 3D features for 3D part segmentation at multiple granularities. Once the segmented parts are obtained from the scale-conditioned part-aware 3D features, we use VLMs to assign semantic labels to each part based on the multi-view renderings. Compared to previous methods, our SAMPart3D can scale to the recent large-scale 3D object dataset Objaverse and handle complex, non-ordinary objects. Additionally, we contribute a new 3D part segmentation benchmark to address the lack of diversity and complexity of objects and parts in existing benchmarks. Experiments show that our SAMPart3D significantly outperforms existing zero-shot 3D part segmentation methods, and can facilitate various applications such as part-level editing and interactive segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。