用低秩适配让SAM2秒懂新类别，少样本分割又快又准。
FS-SAM2: Adapting Segment Anything Model 2 for Few-Shot Semantic Segmentation via Low-Rank Adaptation
- 用低秩适配（LoRA）微调SAM2，仅更新少量参数。
- 在PASCAL-5i等数据集上达领先性能，支持任意K-shot。
- 推理高效，适合快速部署到新场景的分割任务。
少样本语义分割旨在仅用少量标注样本就实现对未见类别的分割。现有方法通常从头训练额外模块，需大量数据和计算资源。本文提出基于SAM2的少样本分割方法FS-SAM2，直接复用SAM2的视频分割能力处理少样本任务，并采用低秩适配（LoRA）优化其原始模块，以应对标准数据集中多样图像分布，而非SAM2预训练时的时序帧。该方法仅需少量参数进行元训练，有效适配同时保留了SAM2的强大分割性能。支持任意K-shot设置，在PASCAL-5$^i$、COCO-20$^i$和FSS-1000数据集上表现优异，且推理效率高。代码已公开于https://github.com/fornib/FS-SAM2。
原文摘要 · Abstract (English)
Few-shot semantic segmentation has recently attracted great attention. The goal is to develop a model capable of segmenting unseen classes using only a few annotated samples. Most existing approaches adapt a pre-trained model by training from scratch an additional module. Achieving optimal performance with these approaches requires extensive training on large-scale datasets. The Segment Anything Model 2 (SAM2) is a foundational model for zero-shot image and video segmentation with a modular design. In this paper, we propose a Few-Shot segmentation method based on SAM2 (FS-SAM2), where SAM2's video capabilities are directly repurposed for the few-shot task. Moreover, we apply a Low-Rank Adaptation (LoRA) to the original modules in order to handle the diverse images typically found in standard datasets, unlike the temporally connected frames used in SAM2's pre-training. With this approach, only a small number of parameters is meta-trained, which effectively adapts SAM2 while benefiting from its impressive segmentation performance. Our method supports any K-shot configuration. We evaluate FS-SAM2 on the PASCAL-5$^i$, COCO-20$^i$ and FSS-1000 datasets, achieving remarkable results and demonstrating excellent computational efficiency during inference. Code is available at https://github.com/fornib/FS-SAM2
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。