arXiv:2503.07890cs.CV2025-03ICCV被引 18

用生成式扩散模型做遥感图像分类,性能超越现有方法。

Can Generative Geospatial Diffusion Models Excel as Discriminative Geospatial Foundation Models?

  • 将生成式扩散模型转为判别式预训练工具,融合多阶段噪声特征。
  • 在语义分割上提升5.7% mIoU,分类任务提升7.9% F1-score。
  • 适合想用生成模型提升遥感判别任务性能的研究者。

自监督学习(SSL)已推动遥感领域表示学习的发展,使地理空间基础模型(GFMs)能够利用海量未标注卫星图像完成多种下游任务。当前的GFMs主要依赖对比学习或掩码图像建模等目标,因其在迁移表征学习中的成功表现。然而,生成式扩散模型在图像生成中展现的多层次语义捕捉能力,在判别任务中仍待探索。本文提出SatDiFuser框架,将基于扩散的生成式地理空间基础模型转化为强大的判别式预训练工具。通过系统分析多阶段、噪声依赖的扩散特征,设计三种融合策略以有效利用这些多样化表示。在多个遥感基准测试中,SatDiFuser的表现优于现有先进GFMs,语义分割任务最高提升5.7% mIoU,分类任务提升7.9% F1-score,证明了生成式扩散模型具备与判别式模型媲美甚至超越的能力。代码开源:https://github.com/yurujaja/SatDiFuser。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) has revolutionized representation learning in Remote Sensing (RS), advancing Geospatial Foundation Models (GFMs) to leverage vast unlabeled satellite imagery for diverse downstream tasks. Currently, GFMs primarily employ objectives like contrastive learning or masked image modeling, owing to their proven success in learning transferable representations. However, generative diffusion models, which demonstrate the potential to capture multi-grained semantics essential for RS tasks during image generation, remain underexplored for discriminative applications. This prompts the question: can generative diffusion models also excel and serve as GFMs with sufficient discriminative power? In this work, we answer this question with SatDiFuser, a framework that transforms a diffusion-based generative geospatial foundation model into a powerful pretraining tool for discriminative RS. By systematically analyzing multi-stage, noise-dependent diffusion features, we develop three fusion strategies to effectively leverage these diverse representations. Extensive experiments on remote sensing benchmarks show that SatDiFuser outperforms state-of-the-art GFMs, achieving gains of up to +5.7% mIoU in semantic segmentation and +7.9% F1-score in classification, demonstrating the capacity of diffusion-based generative foundation models to rival or exceed discriminative GFMs. The source code is available at: https://github.com/yurujaja/SatDiFuser.

遥感扩散模型自监督基础模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。