用雷达图生成云遮掩的卫星光学图像，提升遥感数据可用性
CloudBreaker: Breaking the Cloud Covers of Sentinel-2 Images using Multi-Stage Trained Conditional Flow Matching on Sentinel-1
- 分阶段训练条件流匹配模型，融合余弦调度提升生成质量
- 生成图像FID仅0.7432，植被指数结构相似度超0.68
- 适合缺光/有云区域的遥感分析，如农业监测与灾害评估
云覆盖和夜间条件仍是卫星遥感的主要限制，常导致多光谱影像不可用。相比之下，哨兵-1雷达影像不受云层影响，可全天候获取数据。为解决影像稀缺问题，我们提出CloudBreaker，一种从哨兵-1数据生成高质量哨兵-2多光谱信号的新框架，包括重建光学（RGB）图像及关键植被指数（NDVI）和水体指数（NDWI）。采用基于条件潜在流匹配的新型分阶段训练方法，并首次将余弦调度引入流匹配。CloudBreaker表现优异，生成图像的弗雷歇起始距离（FID）为0.7432，表明高保真与真实感；对NDWI的结构相似性指数（SSIM）达0.6156，对NDVI达0.6874，体现良好结构一致性。该模型为多光谱数据不可靠或缺失的遥感应用提供了有力解决方案。
原文摘要 · Abstract (English)
Cloud cover and nighttime conditions remain significant limitations in satellite-based remote sensing, often restricting the availability and usability of multi-spectral imagery. In contrast, Sentinel-1 radar images are unaffected by cloud cover and can provide consistent data regardless of weather or lighting conditions. To address the challenges of limited satellite imagery, we propose CloudBreaker, a novel framework that generates high-quality multi-spectral Sentinel-2 signals from Sentinel-1 data. This includes the reconstruction of optical (RGB) images as well as critical vegetation and water indices such as NDVI and NDWI. We employed a novel multi-stage training approach based on conditional latent flow matching and, to the best of our knowledge, are the first to integrate cosine scheduling with flow matching. CloudBreaker demonstrates strong performance, achieving a Frechet Inception Distance (FID) score of 0.7432, indicating high fidelity and realism in the generated optical imagery. The model also achieved Structural Similarity Index Measure (SSIM) of 0.6156 for NDWI and 0.6874 for NDVI, indicating a high degree of structural similarity. This establishes CloudBreaker as a promising solution for a wide range of remote sensing applications where multi-spectral data is typically unavailable or unreliable
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。