提出新型融合网络,提升遥感图像超分辨率生成质量。
RAFNet: Region-Aware Fusion Network for Pansharpening

- 用小波变换与聚类动态生成自适应卷积核,实现区域感知特征增强。
- 引入稀疏注意力机制,计算量降低且保留高频细节,性能领先现有方法。
- 适合遥感图像处理、高分影像融合等需要高效精准重建的场景。
全色锐化旨在通过融合低分辨率多光谱(LRMS)和高分辨率全色(PAN)图像,生成高分辨率多光谱(HRMS)图像。尽管深度学习已推动该领域发展,主流基于频率的方法依赖标准缩放点积注意力,存在二次计算复杂度问题,并未能利用遥感图像的固有区域稀疏性。此外,现有空间增强策略通常采用静态卷积核,难以适应PAN与MS图像中复杂的频率与区域变化。为此,我们提出区域感知融合(RAFNet)网络,协同建模空间与频率信息。具体而言,设计了空间自适应精炼(SAR)模块,结合离散小波变换(DWT)进行方向性频率分离,以及K均值聚类进行区域划分,从而动态构建区域特异性自适应卷积核,实现时空频自适应特征增强。同时,提出基于语义聚类引导的稀疏注意力机制的聚类频率聚合(CFA)模块,执行区域感知稀疏注意力,显著减少计算冗余,同时保障高质量频率特征提取。进一步将这些模块集成到渐进式多层次空间-频率网络架构中,促进稳健交互与精确图像重建。在多个基准数据集上的大量实验表明,RAFNet在降分辨率与全分辨率评估中均显著优于当前最优方法。代码已开源:https://github.com/PatrickNod/RAFNet。
原文摘要 · Abstract (English)
Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images. Although deep learning has advanced this field, mainstream frequency-based methods relying on standard scaled dot-product attention suffer from quadratic computational complexity and fail to exploit the inherent regional sparsity of remote sensing imagery. Furthermore, existing spatial enhancement strategies typically employ static convolution kernels, which struggle to adapt to the complex frequency and regional variations of PAN and MS images. To address these bottlenecks, we propose a Region-Aware Fusion (RAFNet) Network that synergistically models spatial and frequency information. Specifically, we design a Spatial Adaptive Refinement (SAR) module that leverages the discrete wavelet transform (DWT) for directional frequency separation and K-means clustering for regional partitioning, which enables the dynamic construction of region-specific adaptive convolution kernels, achieving spatially and frequency-adaptive feature enhancement. Moreover, we introduce a Clustered Frequency Aggregation (CFA) module based on a sparse attention mechanism guided by the semantic clusters, which executes a region-aware sparse attention strategy that drastically reduces computational redundancy while ensuring high-quality frequency feature extraction. In addition we integrated these modules into a progressive, multi-level spatial-frequency network architecture to facilitate robust interaction and accurate image reconstruction. Extensive experiments on multiple benchmark datasets demonstrate that the proposed RAFNet significantly outperforms state-of-the-art pansharpening methods in both reduced- and full-resolution assessments. The code is available at https://github.com/PatrickNod/RAFNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。