arXiv:2412.06243cs.CVeess.IV2024-12CVPR被引 12

通过不确定性感知知识蒸馏,提升全色锐化细节恢复能力。

U-Know-DiffPAN: An Uncertainty-aware Knowledge Distillation Diffusion Framework with Details Enhancement for PAN-Sharpening

  • 用频域注意力捕捉高频特征,指导逆过程学习。
  • 结合小波变换与紧凑向量编码,增强多源信息利用。
  • 通过不确定图引导学生模型聚焦难区域,适合高精度遥感图像处理。

传统的全色锐化方法因难以有效利用高频信息,常导致细节恢复不足。基于扩散的方案也因条件不足,未能充分融合全色(PAN)和低分辨率多光谱(LRMS)输入。为此,我们提出一种不确定性感知的知识蒸馏扩散框架——U-Know-DiffPAN。该框架通过频率选择性注意力机制,使教师模型捕获频率细节,从而实现准确的逆过程学习。通过将编码器条件化为PAN与LRMS的紧凑向量表示,解码器则基于小波变换,实现丰富的频率信息利用。高容量教师模型借助不确定性图,将富含频率的特征蒸馏至轻量化学生模型,指导其在困难区域进行重点优化。大量实验表明,U-Know-DiffPAN在多个数据集上均优于近期最先进方法,具备更强鲁棒性与优越性能。

原文摘要 · Abstract (English)

Conventional methods for PAN-sharpening often struggle to restore fine details due to limitations in leveraging high-frequency information. Moreover, diffusion-based approaches lack sufficient conditioning to fully utilize Panchromatic (PAN) images and low-resolution multispectral (LRMS) inputs effectively. To address these challenges, we propose an uncertainty-aware knowledge distillation diffusion framework with details enhancement for PAN-sharpening, called U-Know-DiffPAN. The U-Know-DiffPAN incorporates uncertainty-aware knowledge distillation for effective transfer of feature details from our teacher model to a student one. The teacher model in our U-Know-DiffPAN captures frequency details through freqeuncy selective attention, facilitating accurate reverse process learning. By conditioning the encoder on compact vector representations of PAN and LRMS and the decoder on Wavelet transforms, we enable rich frequency utilization. So, the high-capacity teacher model distills frequency-rich features into a lightweight student model aided by an uncertainty map. From this, the teacher model can guide the student model to focus on difficult image regions for PAN-sharpening via the usage of the uncertainty map. Extensive experiments on diverse datasets demonstrate the robustness and superior performance of our U-Know-DiffPAN over very recent state-of-the-art PAN-sharpening methods.

全色锐化扩散模型知识蒸馏遥感图像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。