用视觉模型生成伪标注，提升开放世界实例分割的泛化与精度
SOS: Segment Object System for Open-World Instance Segmentation With Object Priors
- 基于SAM和自监督ViT注意力图生成高质量对象提示
- 在COCO、LVIS、ADE20k上实现81.6%精度提升
- 适合需要高精度泛化分割的应用场景
我们提出一种开放世界实例分割（OWIS）方法，旨在从有限标注类别中泛化出任意未知物体的分割结果。针对现有系统泛化能力弱且常误检背景的问题，我们的Segment Object System（SOS）利用基础模型SAM生成高质量伪标注。通过研究多种对象先验，发现来自自监督视觉变换器的自注意力图效果最佳，用于引导SAM聚焦于真实物体。最终，对SAM输出进行后处理得到的分割结果作为伪标注，训练标准实例分割模型。该方法在COCO、LVIS、ADE20k数据集上均展现强泛化能力，相比最先进方法精度最高提升81.6%。代码已开源：https://github.com/chwilms/SOS
原文摘要 · Abstract (English)
We propose an approach for Open-World Instance Segmentation (OWIS), a task that aims to segment arbitrary unknown objects in images by generalizing from a limited set of annotated object classes during training. Our Segment Object System (SOS) explicitly addresses the generalization ability and the low precision of state-of-the-art systems, which often generate background detections. To this end, we generate high-quality pseudo annotations based on the foundation model SAM. We thoroughly study various object priors to generate prompts for SAM, explicitly focusing the foundation model on objects. The strongest object priors were obtained by self-attention maps from self-supervised Vision Transformers, which we utilize for prompting SAM. Finally, the post-processed segments from SAM are used as pseudo annotations to train a standard instance segmentation system. Our approach shows strong generalization capabilities on COCO, LVIS, and ADE20k datasets and improves on the precision by up to 81.6% compared to the state-of-the-art. Source code is available at: https://github.com/chwilms/SOS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。