提出通用遥感图像融合模型,跨卫星跨场景表现更稳定。
Universal Pansharpening Model
- 用可逆光谱基映射任意波段多光谱到统一空间
- 结合扩散模型与像素-潜在交互,实现稳定可控融合
- 适合需要跨卫星、跨场景应用的遥感图像处理任务
全色融合旨在通过融合纹理丰富的全色(PAN)图像和低分辨率多光谱(MS)图像,生成高分辨率多光谱图像。现有方法多依赖特定卫星和具体场景,泛化能力差,实用性受限。为此,我们提出UniPS,一种卫星无关且场景鲁棒的通用全色融合模型。核心是引入模态交错变换器,学习波段特异性表示,构建可逆光谱仿射基,通过张量乘法将任意波段的多光谱映射至统一潜在空间。在此基础上,构建潜在扩散桥模型,逐步演化潜在表示,并引入桥后验采样,将潜在扩散与像素空间观测耦合,实现稳定可控融合。此外,设计无限维像素-潜在交互机制,全面捕捉全色观测与多光谱表示间的跨域依赖,促进互补信息融合。为支持大规模训练与评估,我们构建了涵盖全球多卫星、多场景的全色融合基准数据集PSBench。大量实验表明,UniPS在各类任务中持续优于现有最优方法,展现出卓越的泛化性与鲁棒性。
原文摘要 · Abstract (English)
Pansharpening generates the high-resolution multi-spectral (MS) image by integrating spatial details from a texture-rich panchromatic (PAN) image and spectral attributes from a low-resolution MS image. Existing methods are predominantly satellite-specific and scene-dependent, which severely limits their generalization across heterogeneous sensors and varied scenes, thereby reducing their real-world practicality. To address these challenges, we present UniPS, a universal pansharpening model for satellite-agnostic and scene-robust fusion. Specifically, we introduce a modality-interleaved transformer that learns band-wise modal specializations to form reversible spectral affine bases, mapping arbitrary-band MS into a unified latent space via tensor multiplication. Building upon this, we construct a latent diffusion bridge model to progressively evolve latent representations, and incorporate bridge posterior sampling to couple latent diffusion with pixel-space observations, enabling stable and controllable fusion. Furthermore, we devise infinite-dimensional pixel-to-latent interaction mechanisms to comprehensively capture the cross-domain dependencies between PAN observations and MS representations, thereby facilitating complementary information fusion. In addition, to support large-scale training and evaluation, we construct a comprehensive pansharpening benchmark, termed PSBench, consisting of worldwide MS and PAN image pairs from multiple satellites across diverse scenes. Extensive experiments verify that UniPS consistently outperforms state-of-the-art methods, exhibiting superior generalization and robustness across a wide range of tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。