提升高光谱图像融合效果,用更小模型实现更高清重建。
HyFusion: Enhanced Reception Field Transformer for Hyperspectral Image Fusion
- 设计双耦合网络,让多光谱与低分辨率高光谱图像互相增强特征。
- 引入增强感受野模块,捕捉长距离依赖,减少信息丢失。
- 适合资源受限场景,兼顾精度与计算效率,适合遥感图像处理者。
高光谱图像(HSI)融合旨在从高分辨率多光谱图像(HR-MSI)和低分辨率高光谱图像(LR-HSI)中重建出高分辨率高光谱图像(HR-HSI),该任务因高质量HSI获取成本高、硬件限制大而至关重要。现有方法虽利用空间与光谱关系,但常受限于感受野小、特征利用率低,导致性能不佳。同时,高质量HSI数据稀缺,更需高效利用数据以提升重建质量。为此,本文提出HyFusion,一种新型双耦合网络(DCN)框架,旨在增强跨域特征提取并实现特征图重用。该框架首先通过专用子网络处理HR-MSI与LR-HSI输入,在特征提取过程中相互增强,保留互补的空间与光谱细节。核心在于使用增强感受野块(ERFB),结合移位窗口注意力与密集连接,扩展感受野,有效捕捉长程依赖且最小化信息损失。大量实验表明,HyFusion在HR-MSI/LR-HSI融合任务中达到当前最优性能,显著提升重建质量,同时保持紧凑模型规模与计算效率。通过将增强感受野与特征重用融入耦合网络架构,HyFusion为资源受限场景下的HSI融合提供了一种实用高效方案,树立了新基准。代码将公开。
原文摘要 · Abstract (English)
Hyperspectral image (HSI) fusion addresses the challenge of reconstructing High-Resolution HSIs (HR-HSIs) from High-Resolution Multispectral images (HR-MSIs) and Low-Resolution HSIs (LR-HSIs), a critical task given the high costs and hardware limitations associated with acquiring high-quality HSIs. While existing methods leverage spatial and spectral relationships, they often suffer from limited receptive fields and insufficient feature utilization, leading to suboptimal performance. Furthermore, the scarcity of high-quality HSI data highlights the importance of efficient data utilization to maximize reconstruction quality. To address these issues, we propose HyFusion, a novel Dual-Coupled Network (DCN) framework designed to enhance cross-domain feature extraction and enable effective feature map reusing. The framework first processes HR-MSI and LR-HSI inputs through specialized subnetworks that mutually enhance each other during feature extraction, preserving complementary spatial and spectral details. At its core, HyFusion utilizes an Enhanced Reception Field Block (ERFB), which combines shifting-window attention and dense connections to expand the receptive field, effectively capturing long-range dependencies while minimizing information loss. Extensive experiments demonstrate that HyFusion achieves state-of-the-art performance in HR-MSI/LR-HSI fusion, significantly improving reconstruction quality while maintaining a compact model size and computational efficiency. By integrating enhanced receptive fields and feature map reusing into a coupled network architecture, HyFusion provides a practical and effective solution for HSI fusion in resource-constrained scenarios, setting a new benchmark in hyperspectral imaging. Our code will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。