arXiv:2602.06333cs.CV2026-02被引 3

让SAM3在真实场景中更稳定,自动适应概念变化。

Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation

  • 用目标数据统计构建动态概念库,替代固定提示
  • 在自然场景和遥感图像上显著提升分割鲁棒性
  • 无需参数更新,适合部署在变化环境中的应用

最近提出的SAM3通过可提示的概念分割技术革新了开放词汇分割(OVS),能将像素预测与灵活的概念提示对齐。但其依赖预定义概念,在目标域出现视觉分布偏移(数据漂移)或标签分布演变(概念漂移)时,视觉证据与提示间的对齐会失效。本文提出无参校准框架ConceptBank,实时恢复这种对齐。方法包括:(i) 使用类级视觉原型锚定目标域证据,(ii) 挖掘代表性支持样本以抑制数据漂移下的异常值,(iii) 融合候选概念修正概念漂移。实验证明,ConceptBank有效使SAM3适应各类分布漂移,涵盖复杂自然场景与遥感任务,建立了OVS领域新的鲁棒性与效率基准。代码与模型开源于https://github.com/pgsmall/ConceptBank。

原文摘要 · Abstract (English)

The recent introduction of \texttt{SAM3} has revolutionized Open-Vocabulary Segmentation (OVS) through \textit{promptable concept segmentation}, which grounds pixel predictions in flexible concept prompts. However, this reliance on pre-defined concepts makes the model vulnerable: when visual distributions shift (\textit{data drift}) or conditional label distributions evolve (\textit{concept drift}) in the target domain, the alignment between visual evidence and prompts breaks down. In this work, we present \textsc{ConceptBank}, a parameter-free calibration framework to restore this alignment on the fly. Instead of adhering to static prompts, we construct a dataset-specific concept bank from the target statistics. Our approach (\textit{i}) anchors target-domain evidence via class-wise visual prototypes, (\textit{ii}) mines representative supports to suppress outliers under data drift, and (\textit{iii}) fuses candidate concepts to rectify concept drift. We demonstrate that \textsc{ConceptBank} effectively adapts \texttt{SAM3} to distribution drifts, including challenging natural-scene and remote-sensing scenarios, establishing a new baseline for robustness and efficiency in OVS. Code and model are available at https://github.com/pgsmall/ConceptBank.

开放词汇分割SAM3概念漂移遥感图像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。