arXiv:2504.11154cs.CVeess.IV2025-04被引 2

用扩散模型将雷达图像转为光学图像,填补云层遮挡下的数据空白。

SAR-to-RGB Translation with Latent Diffusion for Earth Observation

  • 采用三种扩散模型架构,从雷达图生成合成光学图。
  • 分类任务中准确率提升,冷扩散法在云层去除上表现良好。
  • 即使视觉质量不高,生成图像仍具实用价值,适合遥感缺图场景。

地球观测卫星如哨兵-1(S1)和哨兵-2(S2)提供互补的遥感数据,但受云层或数据缺失影响,S2图像常不可用。为此,我们提出一种基于扩散模型(DM)的SAR-to-RGB图像转换方法,从SAR输入生成合成光学图像。探索了三种设置:两种使用标准扩散模型(分别无/有类别条件),一种使用冷扩散(Cold Diffusion),先融合S2与S1再去除SAR信号。在下游任务中评估生成图像,包括土地覆盖分类和云层去除。尽管生成图像无法完全复现真实S2数据,但仍具信息价值。结果表明,类别条件可提升分类准确率,而云层去除性能虽未优化但仍具竞争力。有趣的是,尽管感知质量较低,冷扩散方案在土地覆盖分类中表现优异,提示传统量化评估指标可能不足以反映生成图像的实际应用价值。研究证明,扩散模型在光学图像缺失的遥感应用中具有潜力。

原文摘要 · Abstract (English)

Earth observation satellites like Sentinel-1 (S1) and Sentinel-2 (S2) provide complementary remote sensing (RS) data, but S2 images are often unavailable due to cloud cover or data gaps. To address this, we propose a diffusion model (DM)-based approach for SAR-to-RGB translation, generating synthetic optical images from SAR inputs. We explore three different setups: two using Standard Diffusion, which reconstruct S2 images by adding and removing noise (one without and one with class conditioning), and one using Cold Diffusion, which blends S2 with S1 before removing the SAR signal. We evaluate the generated images in downstream tasks, including land cover classification and cloud removal. While generated images may not perfectly replicate real S2 data, they still provide valuable information. Our results show that class conditioning improves classification accuracy, while cloud removal performance remains competitive despite our approach not being optimized for it. Interestingly, despite exhibiting lower perceptual quality, the Cold Diffusion setup performs well in land cover classification, suggesting that traditional quantitative evaluation metrics may not fully reflect the practical utility of generated images. Our findings highlight the potential of DMs for SAR-to-RGB translation in RS applications where RGB images are missing.

遥感图像扩散模型SAR转光学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。