用小模型让大模型精准分割新物体,省时省力
SAMIC: Segment Anything with In-Context Spatial Prompt Engineering
- 设计轻量网络SAMIC,动态生成空间提示引导大模型
- 仅260万参数,训练数据为1/5仍达顶尖水平
- 适合快速部署到医疗、遥感等新场景的图像分割
少样本分割旨在仅用少量标注图像识别特定类别的物体(如飞机)。当前最优方法需为每个新领域构建资源密集型模型,且必须在大量无关物体(如汽车、火车、动物)上预训练以实现知识迁移。本文提出SAMIC,一个可适配现有视觉基础模型(VFMs)的小型网络,通过学习如何生成上下文空间提示,使这些大模型能准确分割新类别物体。该方法将任何任务转化为少样本学习问题。SAMIC仅260万参数,比领先模型(如带ResNet101主干的4500万参数模型)小94%。即使仅使用单样本基准数据的1/5,其在多种少样本与语义分割数据集(包括COCO-20i、Pascal-5i、PerSeg、FSS-1000和NWPU VHR-10)上表现媲美或超越现有方法。
原文摘要 · Abstract (English)
Few-shot segmentation is the problem of learning to identify specific types of objects (e.g., airplanes) in images from a small set of labeled reference images. The current state of the art is driven by resource-intensive construction of models for every new domain-specific application. Such models must be trained on enormous labeled datasets of unrelated objects (e.g., cars, trains, animals) so that their ``knowledge'' can be transferred to new types of objects. In this paper, we show how to leverage existing vision foundation models (VFMs) to reduce the incremental cost of creating few-shot segmentation models for new domains. Specifically, we introduce SAMIC, a small network that learns how to prompt VFMs in order to segment new types of objects in domain-specific applications. SAMIC enables any task to be approached as a few-shot learning problem. At 2.6 million parameters, it is 94% smaller than the leading models (e.g., having ResNet 101 backbone with 45+ million parameters). Even using 1/5th of the training data provided by one-shot benchmarks, SAMIC is competitive with, or sets the state of the art, on a variety of few-shot and semantic segmentation datasets including COCO-$20^i$, Pascal-$5^i$, PerSeg, FSS-1000, and NWPU VHR-10.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。