将可学习矩阵融入传统算法，用合成数据训练出更准的高光谱图像融合模型
SCALMU: Synthetically-trained Coupling of Adaptive Learned Multiplicative Updates for Hyperspectral-Multispectral Fusion
- 在经典非负矩阵分解框架中嵌入自适应可学习矩阵
- 合成数据训练下超越现有方法，提升融合图像质量
- 兼顾物理可解释性，适合需要透明推理的场景
高光谱-多光谱图像（HSI-MSI）融合旨在从低分辨率高光谱图像和高分辨率多光谱图像中恢复高分辨率高光谱图像。传统方法如耦合非负矩阵分解（CNMF）虽具有强物理可解释性，但性能逊于深度学习方法。为此，本文提出SCALMU（Synthetically-trained Coupling of Adaptive Learned Multiplicative Updates），一种新型无监督展开神经网络架构，在CNMF乘法更新框架中引入自适应可学习矩阵，显著提升性能。由于架构与CNMF高度接近，该方法保持了物理可解释性与非负性约束。针对监督数据稀缺问题，采用死叶模型生成合成HSI-MSI数据集，并实现端到端合成监督训练。多个数据集上的实验表明，SCALMU优于当前最优方法，验证了合成数据驱动的无监督融合潜力。代码已开源：https://github.com/xinxinxu99/SCALMU.git
原文摘要 · Abstract (English)
HyperSpectral-MultiSpectral Image (HSI-MSI) fusion aims to recover a high-resolution hyperspectral image from a low-resolution HSI and a high-resolution MSI. Classical methods such as Coupled Nonnegative Matrix Factorization (CNMF) benefit from a strong physical interpretability but suffer from inferior results compared to their deep-learning counterparts. To address this limitation, we propose SCALMU (Synthetically-trained Coupling of Adaptive Learned Multiplicative Updates), a novel blind unrolled neural network architecture that integrates adaptive learnable matrices within the classical framework of CNMF multiplicative updates, improving its results. Due to its architectural proximity with CNMF, the resulting algorithm preserves physical interpretability and nonnegativity constraints. To overcome the scarcity of supervised training data, we generate a synthetic HSI-MSI dataset using the dead leaves model and train SCALMU end-to-end under synthetic supervision. Experiments on several datasets show that SCALMU outperforms state-of-the-art methods and highlights the potential of blind fusion trained with synthetic data. The code is available at https://github.com/xinxinxu99/SCALMU.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。