无需遥感微调，用DINOv3实现开放词汇遥感分割新纪录
DINO Soars: DINOv3 for Open-Vocabulary Semantic Segmentation of Remote Sensing Imagery
- 通过代价聚合与无训练上采样，提升DINOv3在遥感图像的开放词汇分割能力
- 在多个遥感分割数据集上超越需微调的现有方法，达到最新水平
- 仅在COCO-Stuff子集微调，避免遥感领域标注数据依赖
遥感领域缺乏密集标注数据集，获取成本高。因此，无需监督微调即可良好分割遥感图像的模型极具价值，但现有方案仍落后于监督方法。近期DINOv3在未预训练于遥感数据的情况下，超越了现有遥感基础模型在GEO-bench分割基准上的表现。同时，DINO.txt使基于DINOv3主干网络的开放词汇语义分割（OVSS）成为可能。我们利用这些进展，构建了一个无需遥感领域微调的遥感图像开放词汇分割模型：CAFe-DINO（Cost Aggregation + Feature Upsampling with DINO）。该模型通过代价聚合和文本-图像相似度分数的训练自由上采样，发挥DINOv3的强大泛化能力。其鲁棒的潜在表示无需在遥感图像上微调；我们仅在面向遥感的COCO-Stuff子集上进行微调。CAFe-DINO在关键遥感分割数据集上取得当前最优性能，优于在遥感数据上微调的开放词汇分割方法。代码与数据已公开于https://github.com/rfaulk/DINO_Soars。
原文摘要 · Abstract (English)
The remote sensing (RS) domain suffers from a lack of densely labeled datasets, which are costly to obtain. Thus, models that can segment RS imagery well without supervised fine-tuning are valuable, but existing solutions fall behind supervised methods. Recently, DINOv3 surpassed SOTA RS foundation models on the GEO-bench segmentation benchmark without pre-training on RS data. Additionally, DINO.txt has enabled open vocabulary semantic segmentation (OVSS) with the DINOv3 backbone. We leverage these developments to form an OVSS model for RS imagery, free of RS-domain fine-tuning. Our model, CAFe-DINO (Cost Aggregation + Feature Upsampling with DINO) exploits the strong OVSS performance of DINOv3 for RS imagery via cost aggregation and training-free upsampling of text-image similarity scores. The robust latent of the DINOv3 backbone eliminates the need for fine-tuning on RS imagery; we instead fine-tune our model on a RS-targeted subset of COCO-Stuff. CAFe-DINO achieves state-of-the-art performance on key RS segmentation datasets, outperforming OVSS methods fine-tuned on RS data. Our code and data are publicly available at https://github.com/rfaulk/DINO_Soars.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。