提出快速聚类算法，实现无监督图像分割的高效预训练。
S2-UniSeg: Fast Universal Agglomerative Pooling for Scalable Segment Anything without Supervision
- 用并行聚类算法快速生成多粒度伪掩码，毫秒级完成每张图处理。
- 在四个基准上超越当前最优模型，最高提升达11.1%（UVO AR）。
- 适合大规模无监督分割研究，尤其关注效率与可扩展性的团队。
近期自监督图像分割模型在语义分割和无类别实例分割上表现优异，但其预训练流程为多阶段，需在每轮训练间进行耗时的伪掩码生成。这一离线过程不仅难以随数据集规模扩展，且因优化不连续导致次优解。为此，我们提出新型伪掩码生成算法——快速通用聚合池化（UniAP），可并行识别各层相似节点，毫秒内生成语义级、实例级及多粒度伪掩码。基于此，我们构建了可扩展的自监督通用分割模型S2-UniSeg，采用学生-动量教师框架实现连续预训练，并设计面向分割的查询级自蒸馏（QuerySD）任务以学习局部到全局对应关系。在相同设置下，S2-UniSeg在COCO上提升AP+6.9，UVO上提升AR+11.1，COCOStuff-27上提升PixelAcc+4.5，Cityscapes上提升RQ+8.0。在扩大至SA-1B的200万图像子集后，四项指标均进一步提升。代码与预训练模型已开源。
原文摘要 · Abstract (English)
Recent self-supervised image segmentation models have achieved promising performance on semantic segmentation and class-agnostic instance segmentation. However, their pretraining schedule is multi-stage, requiring a time-consuming pseudo-masks generation process between each training epoch. This time-consuming offline process not only makes it difficult to scale with training dataset size, but also leads to sub-optimal solutions due to its discontinuous optimization routine. To solve these, we first present a novel pseudo-mask algorithm, Fast Universal Agglomerative Pooling (UniAP). Each layer of UniAP can identify groups of similar nodes in parallel, allowing to generate both semantic-level and instance-level and multi-granular pseudo-masks within ens of milliseconds for one image. Based on the fast UniAP, we propose the Scalable Self-Supervised Universal Segmentation (S2-UniSeg), which employs a student and a momentum teacher for continuous pretraining. A novel segmentation-oriented pretext task, Query-wise Self-Distillation (QuerySD), is proposed to pretrain S2-UniSeg to learn the local-to-global correspondences. Under the same setting, S2-UniSeg outperforms the SOTA UnSAM model, achieving notable improvements of AP+6.9 on COCO, AR+11.1 on UVO, PixelAcc+4.5 on COCOStuff-27, RQ+8.0 on Cityscapes. After scaling up to a larger 2M-image subset of SA-1B, S2-UniSeg further achieves performance gains on all four benchmarks. Our code and pretrained models are available at https://github.com/bio-mlhui/S2-UniSeg
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。