通过去除参考图背景特征,提升零样本图像分割精度。
REBASE: Reference-Background Subspace Elimination for Training-Free In-Context Segmentation

- 从参考图中识别背景特征子空间并正交投影消除干扰
- 在PACO-Part等数据集上超越现有无训练方法性能
- 适合快速部署新类别分割,无需重新训练
无训练的上下文分割可在推理时仅用一张标注参考图引入新物体类别,避免类别增量学习的重训练与内存开销。现有方法结合视觉基础模型与可提示分割网络(如SAM),但其性能受限于跨图相似性图质量;参考图与查询图间共有的上下文背景会系统性提升非目标区域的相似度,导致提示定位不准。我们提出REBASE,一种无训练框架,显式抑制这些虚假上下文对应。方法从参考图中识别低秩背景特征子空间,并在闭式解下将参考图与查询图特征投影至其正交补空间,实现更清晰的语义匹配。随后采用相似性加权最远点采样生成正样本点提示,并结合优化的密集相似性先验。无需任何训练或参数更新,本方法在PACO-Part、FSS-1000及ISIC2018等跨域数据集上达到无训练方法新最优性能,证明了显式背景子空间消除是单样本定位的有效原则。
原文摘要 · Abstract (English)
Training-free in-context segmentation enables new object categories to be introduced at inference time from a single annotated reference image, eliminating the retraining and memory overhead of class-incremental learning. Recent approaches achieve this by combining vision foundation models for semantic correspondence with promptable segmentation networks like SAM. However, their performance is fundamentally limited by the quality of the cross-image similarity map; shared contextual backgrounds between the reference and query systematically elevate similarity in non-target regions, degrading prompt localization. We present REBASE, a training-free framework that explicitly suppresses these spurious contextual correspondences. Our method identifies the low-rank background feature subspace from the reference image and project the reference and query features onto its orthogonal complement in closed form, yielding cleaner semantic matching. We then generate positive point prompts using similarity-weighted farthest-point sampling, paired with a refined dense similarity prior. Without any training or parameter updates, our approach establishes a new state of the art among training-free methods on PACO-Part, FSS-1000, and cross-domain datasets such as ISIC2018, demonstrating that explicit background subspace removal is a highly effective principle for one-shot localization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。