arXiv:2409.01691cs.CV2024-09被引 8

用稀疏标注实现牙齿点云精准分割,性能接近全监督方法。

When 3D Partial Points Meets SAM: Tooth Point Cloud Segmentation with Sparse Labels

  • 基于SAM的置信度提示生成策略,自动筛选有效点提示。
  • 仅需0.1%标注(每牙一点)即超越现有弱监督方法。
  • 适合标注成本高的口腔医疗3D分割任务,尤其缺数据场景。

牙齿点云分割是众多正畸应用的基础任务。当前研究多依赖昂贵且繁琐的逐点人工标注,采用全监督学习。尽管近期提出弱监督方法使用弱标签进行3D分割并取得良好效果,但在标签极度稀疏时仍表现不佳。受Segment Anything Model(SAM)强大可提示分割能力的启发,本文提出SAMTooth框架,利用其能力补充极稀疏监督。为自动生成合适点提示,提出一种新型置信度感知提示生成策略,通过聚合粗粒度类别预测并结合置信度过滤。此外,为充分利用SAM输出中的结构与形状线索以辅助3D特征学习,设计了掩码引导表示学习,将SAM生成的牙齿掩码重新投影至3D空间,并约束不同牙齿的点具有区分性表征。在公开数据集上的实验表明,仅需0.1%标注(每牙一个点),该方法显著超越近期弱监督方法,性能甚至可媲美最新全监督方法,展现出将SAM应用于稀疏标签3D感知任务的巨大潜力。代码已开源。

原文摘要 · Abstract (English)

Tooth point cloud segmentation is a fundamental task in many orthodontic applications. Current research mainly focuses on fully supervised learning which demands expensive and tedious manual point-wise annotation. Although recent weakly-supervised alternatives are proposed to use weak labels for 3D segmentation and achieve promising results, they tend to fail when the labels are extremely sparse. Inspired by the powerful promptable segmentation capability of the Segment Anything Model (SAM), we propose a framework named SAMTooth that leverages such capacity to complement the extremely sparse supervision. To automatically generate appropriate point prompts for SAM, we propose a novel Confidence-aware Prompt Generation strategy, where coarse category predictions are aggregated with confidence-aware filtering. Furthermore, to fully exploit the structural and shape clues in SAM's outputs for assisting the 3D feature learning, we advance a Mask-guided Representation Learning that re-projects the generated tooth masks of SAM into 3D space and constrains these points of different teeth to possess distinguished representations. To demonstrate the effectiveness of the framework, we conduct experiments on the public dataset and surprisingly find with only 0.1\% annotations (one point per tooth), our method can surpass recent weakly supervised methods by a large margin, and the performance is even comparable to the recent fully-supervised methods, showcasing the significant potential of applying SAM to 3D perception tasks with sparse labels. Code is available at https://github.com/CUHK-AIM-Group/SAMTooth.

点云分割弱监督SAM口腔医学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。