arXiv:2503.16282cs.CV2025-03CVPR被引 25

用视觉语言模型增强少样本3D点云分割,提升新类别泛化能力。

Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model

论文配图:Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model
图 1 · 摘自论文原文
  • 利用视觉语言模型生成伪标签,结合少量样本精炼特征原型。
  • 在多个数据集上达到新纪录,显著提升对未见类别的分割精度。
  • 适合研究少样本3D场景理解与跨模态学习的开发者参考。

通用少样本3D点云分割(GFS-PCS)旨在用少量支持样本适应新类别,同时保持基础类别的分割性能。现有方法通过交互支持或查询特征来增强原型,但受限于少样本带来的稀疏知识。3D视觉语言模型(3D VLMs)虽具备丰富的开放世界新类别知识,却存在噪声问题。本文提出协同利用3D VLM的稠密但含噪伪标签与精确但稀疏的少样本数据的框架GFS-VL。首先采用原型引导的伪标签筛选策略,去除低质量区域;随后设计自适应填充机制,融合伪标签上下文与少样本信息,对筛选后的未标注区域进行精准标注。此外,引入新颖类混合策略,将少样本嵌入训练场景以保留关键上下文,促进新类别学习。针对当前基准多样性不足的问题,构建两个包含多样化新类别的挑战性评估基准。实验验证了该框架在多种模型和数据集上的有效性,为真实世界中的GFS-PCS发展奠定基础。代码已开源。

原文摘要 · Abstract (English)

Generalized few-shot 3D point cloud segmentation (GFS-PCS) adapts models to new classes with few support samples while retaining base class segmentation. Existing GFS-PCS methods enhance prototypes via interacting with support or query features but remain limited by sparse knowledge from few-shot samples. Meanwhile, 3D vision-language models (3D VLMs), generalizing across open-world novel classes, contain rich but noisy novel class knowledge. In this work, we introduce a GFS-PCS framework that synergizes dense but noisy pseudo-labels from 3D VLMs with precise yet sparse few-shot samples to maximize the strengths of both, named GFS-VL. Specifically, we present a prototype-guided pseudo-label selection to filter low-quality regions, followed by an adaptive infilling strategy that combines knowledge from pseudo-label contexts and few-shot samples to adaptively label the filtered, unlabeled areas. Additionally, we design a novel-base mix strategy to embed few-shot samples into training scenes, preserving essential context for improved novel class learning. Moreover, recognizing the limited diversity in current GFS-PCS benchmarks, we introduce two challenging benchmarks with diverse novel classes for comprehensive generalization evaluation. Experiments validate the effectiveness of our framework across models and datasets. Our approach and benchmarks provide a solid foundation for advancing GFS-PCS in the real world. The code is at https://github.com/ZhaochongAn/GFS-VL

3D分割少样本学习视觉语言模型点云处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。