无需手工设计规则,直接从自监督点变换器中挖掘出3D物体实例分割信号。
Emergent 3D Instance Segmentation from Self-Supervised Point Transformers

- 用自监督点变换器的注意力查询与键值构建相似性图进行实例分组。
- 在三个数据集上超越仅用输出特征的基线方法,提升显著。
- 适合对免训练、无先验3D分割感兴趣的研究者或工程应用。
无监督的室外激光雷达扫描3D实例分割传统上依赖于手工设计的几何先验,如基于密度的聚类、运动线索或投影2D检测。本文探究一个冻结的自监督点变换器是否已包含分离物体实例所需的结构信息,而无需任何手工几何先验。仅将其作为特征提取器,我们分析了其在SemanticKITTI、nuScenes和Waymo Perception数据集上的内部表示。研究得出四个核心发现:(1) 实例信号集中在注意力查询与键中,而非值或最终输出特征;(2) 输出特征存在语义坍缩,将相邻同类别物体合并,而查询与键仍能保持区分;(3) 该信号在深度上呈双峰分布,最强烈出现在最浅层与最深层编码阶段;(4) 该信号主要由旋转位置编码(RoPE)驱动,移除后其优势消失。基于这些发现,我们提出TokenGraph3D方法——一种无需训练的分组器,通过键相似性图上的连通组件对点进行聚类,不依赖密度聚类或邻近性先验。在相同无先验条件下,显著优于仅使用输出特征的基线方法,使涌现的3D实例结构变得可见。
原文摘要 · Abstract (English)
Unsupervised 3D instance segmentation of outdoor LiDAR scans has traditionally relied on handcrafted geometric priors such as density-based clustering, motion cues, or projected 2D detections. In this work, we investigate whether a frozen, self-supervised point transformer already contains the structural information required to isolate object instances without any handcrafted geometric prior. Using this transformer purely as a feature extractor, we probe its internal representations across the SemanticKITTI, nuScenes, and Waymo Perception datasets. Our analysis yields four core insights: (1) the instance signal concentrates in the attention queries and keys rather than in the values or final output features; (2) output features semantically collapse, merging adjacent same-class objects that the queries and keys keep distinct; (3) this instance signal is bimodal in depth, strongest at the shallowest and deepest encoder stages; and (4) this signal is driven predominantly by the rotary position encoding (RoPE), whose removal collapses its advantage. We put these findings into our method TokenGraph3D, a training-free segmenter that groups points via connected components on a key-similarity graph, using neither density-based clustering nor proximity priors. Under identical prior-free conditions, we substantially outperform output-feature baselines, making the emergent 3D instance structure visible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。