arXiv:2508.00453cs.CV2025-08AAAI被引 3

用先验引导融合,解决多光谱与高光谱图像融合的病态问题。

PIF-Net: Ill-Posed Prior Guided Multispectral and Hyperspectral Image Fusion via Invertible Mamba and Fusion-Aware LoRA

  • 引入病态先验约束,提升融合稳定性
  • 基于可逆Mamba架构实现高效特征保持
  • 轻量级模块动态调优,适合资源受限场景

多光谱与高光谱图像融合(MHIF)的目标是生成兼具丰富光谱信息和精细空间细节的高质量图像。然而,由于光谱与空间信息之间固有的权衡关系以及观测数据的有限性,该任务本质上是病态的。以往研究未能有效处理由数据错位引发的病态性。为此,本文提出PIF-Net框架,显式引入病态先验以有效融合多光谱与高光谱图像。为平衡全局光谱建模与计算效率,设计基于可逆Mamba架构的方法,在特征变换与融合过程中保持信息一致性,确保稳定梯度流与过程可逆性。此外,提出新型融合感知低秩适配模块(Fusion-Aware LoRA),在保持模型轻量化的同时动态校准光谱与空间特征。在多个基准数据集上的大量实验表明,PIF-Net在图像重建性能上显著优于当前最先进方法,同时保持模型高效。

原文摘要 · Abstract (English)

The goal of multispectral and hyperspectral image fusion (MHIF) is to generate high-quality images that simultaneously possess rich spectral information and fine spatial details. However, due to the inherent trade-off between spectral and spatial information and the limited availability of observations, this task is fundamentally ill-posed. Previous studies have not effectively addressed the ill-posed nature caused by data misalignment. To tackle this challenge, we propose a fusion framework named PIF-Net, which explicitly incorporates ill-posed priors to effectively fuse multispectral images and hyperspectral images. To balance global spectral modeling with computational efficiency, we design a method based on an invertible Mamba architecture that maintains information consistency during feature transformation and fusion, ensuring stable gradient flow and process reversibility. Furthermore, we introduce a novel fusion module called the Fusion-Aware Low-Rank Adaptation module, which dynamically calibrates spectral and spatial features while keeping the model lightweight. Extensive experiments on multiple benchmark datasets demonstrate that PIF-Net achieves significantly better image restoration performance than current state-of-the-art methods while maintaining model efficiency.

图像融合可逆网络轻量模型多光谱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。