arXiv:2410.10777cs.CV2024-10TPAMI被引 134

用更强大的ViT模型提升半监督语义分割效果,简单升级胜过复杂设计。

UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation

  • 改用DINOv2等大模型作为编码器,替代老旧ResNet
  • 在ADE20K上达59.3% mIoU,优于旧方法且训练成本更低
  • 适合追求高精度与高效训练的视觉研究者

半监督语义分割旨在利用廉价的无标签图像提升分割能力。现有工作虽通过弱-强一致性正则化显著进步,但普遍仍使用小规模ImageNet-1K预训练的ResNet编码器,并在Pascal、Cityscapes等简单数据集上评估。本文主张将基线切换至大规模预训练的ViT模型(如DINOv2),仅更换编码器即可带来比精细方法设计更大的提升。基于此更强基线,我们提出简化版UniMatch V2,继承V1核心思想,训练成本更低,性能更优。同时指出当前数据集已趋于饱和,应转向ADE20K、COCO等更具挑战性的复杂分类任务。代码、模型及日志均开源。

原文摘要 · Abstract (English)

Semi-supervised semantic segmentation (SSS) aims at learning rich visual knowledge from cheap unlabeled images to enhance semantic segmentation capability. Among recent works, UniMatch improves its precedents tremendously by amplifying the practice of weak-to-strong consistency regularization. Subsequent works typically follow similar pipelines and propose various delicate designs. Despite the achieved progress, strangely, even in this flourishing era of numerous powerful vision models, almost all SSS works are still sticking to 1) using outdated ResNet encoders with small-scale ImageNet-1K pre-training, and 2) evaluation on simple Pascal and Cityscapes datasets. In this work, we argue that, it is necessary to switch the baseline of SSS from ResNet-based encoders to more capable ViT-based encoders (e.g., DINOv2) that are pre-trained on massive data. A simple update on the encoder (even using 2x fewer parameters) can bring more significant improvement than careful method designs. Built on this competitive baseline, we present our upgraded and simplified UniMatch V2, inheriting the core spirit of weak-to-strong consistency from V1, but requiring less training cost and providing consistently better results. Additionally, witnessing the gradually saturated performance on Pascal and Cityscapes, we appeal that we should focus on more challenging benchmarks with complex taxonomy, such as ADE20K and COCO datasets. Code, models, and logs of all reported values, are available at https://github.com/LiheYoung/UniMatch-V2.

半监督语义分割ViTDINOv2

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。