无需训练即可实现多类少样本语义分割,自动定位目标区域。
Segment Any Class (SAC): Multi-Class Few-Shot Semantic Segmentation via Class Region Proposals
- 通过类区域提议生成类相关提示,无需额外训练
- 在COCO-20i上超越现有方法,高类别场景表现更优
- 适合快速适应新任务的少样本场景,无模型修改
Segment-Anything Model (SAM) 是一种基于提示驱动框架的分割基础模型,可生成与类别无关的掩码。然而,将SAM用于自动分割特定类别时,通常需要额外训练。本文提出Segment Any Class (SAC),一种无需训练的多类少样本语义分割方法。SAC在查询图像上生成类区域提议(CRP),从而在可能的目标实例位置自动生成类感知提示。CRP基于类内与类间特征差异生成,无需额外训练。该方法可支持任意N-way K-shot配置,相较于梯度学习式微调,避免了泛化能力损失和灾难性遗忘问题。在COCO-20i基准上,SAC显著优于现有方法,尤其在高N-way场景下表现突出。SAC展示了仅通过提示机制即可在不修改基础模型的前提下,高效适应新任务,具有抗概念丢失、快速在线适配等优势。
原文摘要 · Abstract (English)
The Segment-Anything Model (SAM) is a vision foundation model for segmentation with a prompt-driven framework. SAM generates class-agnostic masks based on user-specified instance-referring prompts. However, adapting SAM for automated segmentation -- where manual input is absent -- of specific object classes often requires additional model training. We present Segment Any Class (SAC), a novel, training-free approach that task-adapts SAM for Multi-class segmentation. SAC generates Class-Region Proposals (CRP) on query images which allows us to automatically generate class-aware prompts on probable locations of class instances. CRPs are derived from elementary intra-class and inter-class feature distinctions without any additional training. Our method is versatile, accommodating any N-way K-shot configurations for the multi-class few-shot semantic segmentation (FSS) task. Unlike gradient-learning adaptation of generalist models which risk the loss of generalization and potentially suffer from catastrophic forgetting, SAC solely utilizes automated prompting and achieves superior results over state-of-the-art methods on the COCO-20i benchmark, particularly excelling in high N-way class scenarios. SAC is an interesting demonstration of a prompt-only approach to adapting foundation models for novel tasks with small, limited datasets without any modifications to the foundation model itself. This method offers interesting benefits such as intrinsic immunity to concept or feature loss and rapid, online task adaptation of foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。