用一步扩散+端到端网络，快速生成高分辨率多光谱图像。
Fose: Fusion of One-Step Diffusion and End-to-End Network for Pansharpening
- 用一步蒸馏压缩扩散模型推理过程，从50步减到1步。
- 在三个基准上表现优于现有方法，速度提升7.42倍。
- 适合需要快速、高质量图像融合的遥感应用。
全色锐化是一项重要图像融合任务，旨在将低分辨率多光谱图像（LRMSI）与高分辨率全色图像（PAN）融合，生成高分辨率多光谱图像（HRMSI）。扩散模型（DM）通过多步去噪实现对LRMSI与HRMSI间残差的精准估计，但计算量大、耗时长。端到端模型（E2E model）虽结构简单，但性能受限于先验知识不足。本文提出一种新型四阶段训练策略，构建轻量级网络Fose，融合一步扩散与端到端模型。通过对增强版先进扩散模型进行一步蒸馏，将推理步骤从50步压缩至仅1步。随后，通过轻量级集成模块融合E2E模型与一步扩散模型。大量实验表明，Fose在三个常用基准上均有显著提升。相比基线扩散模型，速度提升7.42倍，同时性能更优。代码与模型已开源：https://github.com/Kai-Liu001/Fose。
原文摘要 · Abstract (English)
Pansharpening is a significant image fusion task that fuses low-resolution multispectral images (LRMSI) and high-resolution panchromatic images (PAN) to obtain high-resolution multispectral images (HRMSI). The development of the diffusion models (DM) and the end-to-end models (E2E model) has greatly improved the frontier of pansharping. DM takes the multi-step diffusion to obtain an accurate estimation of the residual between LRMSI and HRMSI. However, the multi-step process takes large computational power and is time-consuming. As for E2E models, their performance is still limited by the lack of prior and simple structure. In this paper, we propose a novel four-stage training strategy to obtain a lightweight network Fose, which fuses one-step DM and an E2E model. We perform one-step distillation on an enhanced SOTA DM for pansharping to compress the inference process from 50 steps to only 1 step. Then we fuse the E2E model with one-step DM with lightweight ensemble blocks. Comprehensive experiments are conducted to demonstrate the significant improvement of the proposed Fose on three commonly used benchmarks. Moreover, we achieve a 7.42 speedup ratio compared to the baseline DM while achieving much better performance. The code and model are released at https://github.com/Kai-Liu001/Fose.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。