无需模板或名称，用3D特征场实现通用零件分割与对齐。
PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond
- 通过对比学习融合2D/3D标注与无监督图像，训练连续特征场。
- 比现有方法快数十倍，零件分割准确率最高提升20%。
- 支持跨形状一致性，适用于共分割与对应关系任务。
我们提出PartField，一种前馈式3D部件特征学习方法，捕捉部件及其层级的通用概念，不依赖预定义模板或文本名称，可应用于多种模态的开放世界3D形状。PartField推理时仅需一次3D前馈，显著提升运行效率和鲁棒性。模型通过对比学习，从标注数据集与大规模无监督图像分割中蒸馏2D和3D部件提议进行训练，生成连续特征场，可通过聚类获得分层部件分解。实验表明，PartField在准确率上比其他近期无类别零件分割方法最高提升20%，速度常快数个数量级。除单形状部件分解外，所学特征场在不同形状间表现出一致性，支持共分割与对应关系等任务，我们在多个应用中验证了这些通用、分层且一致的3D特征场的有效性。
原文摘要 · Abstract (English)
We propose PartField, a feedforward approach for learning part-based 3D features, which captures the general concept of parts and their hierarchy without relying on predefined templates or text-based names, and can be applied to open-world 3D shapes across various modalities. PartField requires only a 3D feedforward pass at inference time, significantly improving runtime and robustness compared to prior approaches. Our model is trained by distilling 2D and 3D part proposals from a mix of labeled datasets and image segmentations on large unsupervised datasets, via a contrastive learning formulation. It produces a continuous feature field which can be clustered to yield a hierarchical part decomposition. Comparisons show that PartField is up to 20% more accurate and often orders of magnitude faster than other recent class-agnostic part-segmentation methods. Beyond single-shape part decomposition, consistency in the learned field emerges across shapes, enabling tasks such as co-segmentation and correspondence, which we demonstrate in several applications of these general-purpose, hierarchical, and consistent 3D feature fields. Check our Webpage! https://research.nvidia.com/labs/toronto-ai/partfield-release/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。