arXiv:2608.29917cs.CV2026-08

用SAM2统一解决个性化分割与检索,一次对齐搞定两项任务。

FoundYou: A Unified Model for Personalized Segmentation and Retrieval

论文配图:FoundYou: A Unified Model for Personalized Segmentation and Retrieval
图 1 · 摘自论文原文
  • 基于SAM2的实例级一致性,跨图像匹配物体实现统一建模。
  • 在PerMIS和ILIAS上分别提升18.4 mIoU和17.8 mAP。
  • 仅加590万参数,模型更小更快,支持少样本和灵活提示。

个性化分割与个性化检索均旨在识别不同图像中的同一物理对象:前者定位目标图像中的物体,后者检索其出现的图像。尽管目标一致,两者长期独立发展,采用不同方法。本文提出FoundYou,基于观察——训练于视频帧间保持物体身份的Segment Anything 2(SAM 2)天然捕捉实例级线索——利用该特性在独立图像间匹配物体,使分割与检索成为同一实例对齐过程的两个结果。此统一视角带来新能力,包括少样本个性化检索与可提示的个性化分割。大量实验表明,在统一与专用方法上均取得显著提升:PerMIS上提升+18.4 mIoU,ILIAS上提升+17.8 mAP。性能随参考图像增加而提升,对弱提示仍具鲁棒性。超越个性化任务,其在类别级检索基准上也达到顶尖水平。本方法完全冻结SAM 2-small,仅新增590万可训练参数,构建出总参数量5200万的模型,较此前唯一统一方案小20倍、快75倍以上。代码已开源。

原文摘要 · Abstract (English)

Personalized segmentation and personalized retrieval both aim to identify the same physical object across different images. While the former localizes the object within a target image, the latter retrieves images where it appears. Despite this shared instance-level objective, the two tasks have largely evolved separately and are addressed with distinct solutions. In this work, we introduce FoundYou, a unified framework built on the observation that Segment Anything 2 (SAM 2), trained to preserve object identity across video frames, inherently captures instance-level cues. We leverage this property to match objects across independent images, enabling segmentation and retrieval to emerge as two outcomes of the same instance alignment process. This unified view unlocks new capabilities beyond traditional benchmarks, including few-shot personalized retrieval and promptable personalized segmentation with flexible prompts. Extensive experiments show consistent gains over unified and task-specific methods, including +18.4 mIoU on PerMIS and +17.8 mAP on ILIAS. Performance scales with additional references and remains robust to weaker prompts. Beyond personalization, FoundYou achieves state-of-the-art results on category-level retrieval benchmarks. Notably, our approach keeps the SAM 2-small model entirely frozen and adds only 5.9 M trainable parameters, yielding a 52 M-parameter model that is over 75x faster and 20x smaller than the only prior unified solution. Code is available at https://github.com/ga1i13o/FoundYou .

实例分割图像检索统一框架SAM2

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。