首个直接在3D数据上训练的可提示零件分割模型,支持精准识别与自动分解。
PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data
- 采用三平面双分支编码器生成结构化特征,实现可扩展的零件感知表示
- 基于五百万3D形状-零件对训练,支持单提示精准分割与全零件自动分解
- 突破2D迁移局限,能识别表面与内部结构,适合3D理解与生成任务
将3D物体分割为部件是计算机视觉中的长期挑战。为克服分类体系限制并泛化至未见3D对象,近期研究转向开放世界零件分割。这些方法通常通过多视角掩码提升来自2D基础模型(如SAM)的监督,但间接范式难以捕捉内在几何结构,导致仅理解表面、分解不可控且泛化能力有限。我们提出PartSAM,首个在大规模3D数据上原生训练的可提示零件分割模型。遵循SAM的设计理念,PartSAM采用编码器-解码器架构,其三平面双分支编码器生成空间结构化标记,实现可扩展的零件感知表示学习。为实现大规模标注,我们引入模型内循环标注流程,从在线资源中构建超过五百万个3D形状-零件对,提供丰富且细粒度的标签。该架构与多样3D数据的结合催生出涌现的开放世界能力:仅需一个提示,PartSAM即可实现高精度零件识别;在“逐部分分割”模式下,能自动分解形状为表面与内部结构。大量实验表明,PartSAM在多个基准上大幅超越现有最先进方法,标志着迈向3D零件理解基础模型的关键一步。
原文摘要 · Abstract (English)
Segmenting 3D objects into parts is a long-standing challenge in computer vision. To overcome taxonomy constraints and generalize to unseen 3D objects, recent works turn to open-world part segmentation. These approaches typically transfer supervision from 2D foundation models, such as SAM, by lifting multi-view masks into 3D. However, this indirect paradigm fails to capture intrinsic geometry, leading to surface-only understanding, uncontrolled decomposition, and limited generalization. We present PartSAM, the first promptable part segmentation model trained natively on large-scale 3D data. Following the design philosophy of SAM, PartSAM employs an encoder-decoder architecture in which a triplane-based dual-branch encoder produces spatially structured tokens for scalable part-aware representation learning. To enable large-scale supervision, we further introduce a model-in-the-loop annotation pipeline that curates over five million 3D shape-part pairs from online assets, providing diverse and fine-grained labels. This combination of scalable architecture and diverse 3D data yields emergent open-world capabilities: with a single prompt, PartSAM achieves highly accurate part identification, and in a Segment-Every-Part mode, it automatically decomposes shapes into both surface and internal structures. Extensive experiments show that PartSAM outperforms state-of-the-art methods by large margins across multiple benchmarks, marking a decisive step toward foundation models for 3D part understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。