arXiv:2507.02798cs.CV2025-07被引 9

无需训练,仅用参考图即可实现精准实例分割。

No time to train! Training-Free Reference-Based Instance Segmentation

  • 利用基础模型的语义先验,通过参考图与目标图匹配定位对象。
  • 在COCO、PASCAL VOC等数据集上达到36.8%至71.2%的分割精度。
  • 适合快速部署、无标注数据场景,尤其适用于小样本和跨域任务。

图像分割模型的性能长期受限于大规模标注数据的高昂成本。虽然分割一切模型(SAM)通过可提示、语义无关的分割范式缓解了这一问题,但仍需手动视觉提示或复杂的领域相关提示生成规则来处理新图像。为减轻这一负担,本文研究仅提供少量参考图像时的物体分割任务。核心思路是利用基础模型学习到的强大语义先验,在参考图与目标图间识别对应区域。实验表明,这种对应关系可自动生成实例级分割掩码,适用于下游任务。我们提出一种多阶段、无需训练的方法,包括(1)记忆库构建;(2)表征聚合;(3)语义感知特征匹配。在COCO FSOD上取得36.8% nAP,PASCAL VOC少样本任务达71.2% nAP50,跨域少样本分割基准上优于现有无需训练方法,达22.4% nAP。

原文摘要 · Abstract (English)

The performance of image segmentation models has historically been constrained by the high cost of collecting large-scale annotated data. The Segment Anything Model (SAM) alleviates this original problem through a promptable, semantics-agnostic, segmentation paradigm and yet still requires manual visual-prompts or complex domain-dependent prompt-generation rules to process a new image. Towards reducing this new burden, our work investigates the task of object segmentation when provided with, alternatively, only a small set of reference images. Our key insight is to leverage strong semantic priors, as learned by foundation models, to identify corresponding regions between a reference and a target image. We find that correspondences enable automatic generation of instance-level segmentation masks for downstream tasks and instantiate our ideas via a multi-stage, training-free method incorporating (1) memory bank construction; (2) representation aggregation and (3) semantic-aware feature matching. Our experiments show significant improvements on segmentation metrics, leading to state-of-the-art performance on COCO FSOD (36.8% nAP), PASCAL VOC Few-Shot (71.2% nAP50) and outperforming existing training-free approaches on the Cross-Domain FSOD benchmark (22.4% nAP).

实例分割少样本无需训练参考图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。