用参考图提升遥感图像超分辨,解决真实场景中信息失真和依赖过强问题。
Controllable Reference Guided Diffusion with Local Global Fusion for Real World Remote Sensing Image Super Resolution
- 通过双分支融合机制,动态结合参考图的局部与全局特征。
- 在真实数据集上达到当前最优性能,显著提升下游任务效果。
- 支持推理时调节参考强度,适合需要交互控制的遥感应用。
超分辨率技术可提升遥感图像的空间分辨率,助力大规模地球观测。单图超分方法忽略辅助数据中的互补信息,而基于参考图的超分可视为信息融合任务,将历史高分辨率参考图与当前低分辨率观测图结合。然而,现有方法在真实场景下表现不佳,如跨传感器分辨率差异大、地表覆盖变化显著,常导致生成内容不足或过度依赖参考图。为此,本文提出CRefDiff,一种可控的参考引导扩散模型,用于真实世界遥感图像超分辨。为缓解生成不足问题,CRefDiff利用强大生成先验恢复精确结构与纹理;为减少对参考图的过度依赖,引入双分支融合机制,自适应融合参考图的局部与全局信息。该设计还支持推理时调节参考强度,提升模型交互性与灵活性。此外,提出Better Start策略,显著减少去噪步骤,加速推理过程。为推动研究,构建新数据集RealRefRSSRD,包含具有多样地表变化与显著时序差距的高分辨率NAIP与低分辨率Sentinel-2图像对。大量实验表明,CRefDiff在RealRefRSSRD上实现当前最优性能,并有效提升下游任务表现。
原文摘要 · Abstract (English)
Super resolution techniques can enhance the spatial resolution of remote sensing images, enabling more efficient large scale earth observation applications. While single image SR methods enhance low resolution images, they neglect valuable complementary information from auxiliary data. Reference based SR can be interpreted as an information fusion task, where historical high resolution reference images are combined with current LR observations. However, existing RefSR methods struggle with real world complexities, such as cross sensor resolution gap and significant land cover changes, often leading to under generation or over reliance on reference image. To address these challenges, we propose CRefDiff, a novel controllable reference guided diffusion model for real world remote sensing image SR. To address the under generation problem, CRefDiff leverages a powerful generative prior to produce accurate structures and textures. To mitigate over reliance on the reference, we introduce a dual branch fusion mechanism that adaptively fuse both local and global information from the reference image. Moreover, the dual branch design enables reference strength control during inference, enhancing the models interactivity and flexibility. Finally, the Better Start strategy is proposed to significantly reduce the number of denoising steps, thereby accelerating the inference process. To support further research, we introduce RealRefRSSRD, a new real world RefSR dataset for remote sensing images, consisting of HR NAIP and LR Sentinel2 image pairs with diverse land cover changes and significant temporal gaps. Extensive experiments on RealRefRSSRD show that CRefDiff achieves SOTA performance and improves downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。