通过融合频率域信息,提升任意尺度超分辨率性能。
Frequency-Integrated Transformer for Arbitrary-Scale Super-Resolution
- 引入频域信息增强细节表征,无损融合傅里叶特征。
- 在多个数据集上优于现有方法,显著提升高频细节还原能力。
- 适合需要高保真图像重建的视觉任务研究者参考。
基于隐式神经表示的方法在任意尺度超分辨率(ASSR)任务中表现优异,但忽略了频域的潜在价值,导致性能受限。本文提出新型网络频率集成变压器(FIT),通过频域信息融合与利用提升ASSR性能。FIT采用频域融合模块(FIM),结合快速傅里叶变换(FFT)与实虚部映射,无损引入频域信息;频域利用自注意力模块(FUSAM)则通过子空间内空间-频率交互(IISA)和频域相关性计算(FCSA),实现跨域协同与全局上下文捕捉。实验表明,FIT在多个基准数据集上均优于现有方法。可视化特征图验证了FIM在细节刻画上的优势;频域误差图显示IISA有效提升频域保真度;局部归因图证实FCSA能有效捕获全局上下文。
原文摘要 · Abstract (English)
Methods based on implicit neural representation have demonstrated remarkable capabilities in arbitrary-scale super-resolution (ASSR) tasks, but they neglect the potential value of the frequency domain, leading to sub-optimal performance. We proposes a novel network called Frequency-Integrated Transformer (FIT) to incorporate and utilize frequency information to enhance ASSR performance. FIT employs Frequency Incorporation Module (FIM) to introduce frequency information in a lossless manner and Frequency Utilization Self-Attention module (FUSAM) to efficiently leverage frequency information by exploiting spatial-frequency interrelationship and global nature of frequency. FIM enriches detail characterization by incorporating frequency information through a combination of Fast Fourier Transform (FFT) with real-imaginary mapping. In FUSAM, Interaction Implicit Self-Attention (IISA) achieves cross-domain information synergy by interacting spatial and frequency information in subspace, while Frequency Correlation Self-attention (FCSA) captures the global context by computing correlation in frequency. Experimental results demonstrate FIT yields superior performance compared to existing methods across multiple benchmark datasets. Visual feature map proves the superiority of FIM in enriching detail characterization. Frequency error map validates IISA productively improve the frequency fidelity. Local attribution map validates FCSA effectively captures global context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。