无需训练，用DINOv3直接实现多粒度零样本分割
INSID3: Training-Free In-Context Segmentation with DINOv3
- 仅用冻结的DINOv3特征，通过空间结构匹配实现分割
- 在单次示例下达到7.5% mIoU提升，参数量减少3倍
- 适合需要快速部署且不支持标注数据的场景
上下文分割（ICS）旨在仅凭一个带标注的视觉样例，分割任意概念，如物体、部件或个性化实例。现有方法依赖微调视觉基础模型（VFMs），虽提升域内性能但损害泛化能力；或组合多个冻结的VFMs，保持泛化性但导致架构复杂和固定分割粒度。本文从极简视角重新审视ICS，提出：单一自监督主干能否在无任何监督和辅助模型的情况下，同时实现语义匹配与分割？我们发现，缩放后的密集自监督特征（来自DINOv3）具备强空间结构与语义对应性。为此提出INSID3——一种无需训练的方法，仅使用冻结的DINOv3特征，即可根据上下文样例实现可变粒度分割。INSID3在单次语义、部件及个性化分割任务中均达当前最优，相比之前工作提升7.5% mIoU，且参数量减少3倍，无需掩码或类别级监督。代码已开源。
原文摘要 · Abstract (English)
In-context segmentation (ICS) aims to segment arbitrary concepts, e.g., objects, parts, or personalized instances, given one annotated visual examples. Existing work relies on (i) fine-tuning vision foundation models (VFMs), which improves in-domain results but harms generalization, or (ii) combines multiple frozen VFMs, which preserves generalization but yields architectural complexity and fixed segmentation granularities. We revisit ICS from a minimalist perspective and ask: Can a single self-supervised backbone support both semantic matching and segmentation, without any supervision or auxiliary models? We show that scaled-up dense self-supervised features from DINOv3 exhibit strong spatial structure and semantic correspondence. We introduce INSID3, a training-free approach that segments concepts at varying granularities only from frozen DINOv3 features, given an in-context example. INSID3 achieves state-of-the-art results across one-shot semantic, part, and personalized segmentation, outperforming previous work by +7.5 % mIoU, while using 3x fewer parameters and without any mask or category-level supervision. Code is available at https://github.com/visinf/INSID3 .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。