arXiv:2501.01262cs.CV2025-01AAAI被引 25

用Mamba结构提升光谱成像重建精度,细节更清晰。

Detail Matters: Mamba-Inspired Joint Unfolding Network for Snapshot Spectral Compressive Imaging

  • 结合物理模型与学习方法,改进展开网络结构
  • 在模拟和真实数据上均实现更高细节还原度
  • 适合需要高精度光谱图像的科研与工业应用

在编码孔径快照光谱成像系统中,深度展开网络(DUNs)已在单次二维测量下恢复三维高光谱图像(HSIs)方面取得显著进展。然而,由于高光谱图像重建固有的非线性和不适定特性,现有方法在准确性和稳定性上仍面临挑战。为此,我们提出一种受Mamba启发的联合展开网络(MiJUN),将物理嵌入式DUNs与基于学习的成像方法相结合。首先,利用梯形离散化扩展展开网络的表示空间,提出一种加速展开方案,可解释为带二阶微分方程的广义加速半二次分裂,降低对初始优化阶段的依赖,并缓解长程交互问题。关键在于,在Mamba框架内,通过引入选择性状态空间模型和注意力机制,重构了全局到局部注意力机制,将Mamba重新诠释为Transformer的一种变体,提升其适应性和效率。此外,通过将张量模-k展开融入Mamba网络,优化扫描策略,强调各模式下的张量低秩特性,同时方便支持12种扫描方向。在模拟和真实数据集上的数值与视觉对比表明,所提方法具有明显优势,实现了卓越的细节表征能力。

原文摘要 · Abstract (English)

In the coded aperture snapshot spectral imaging system, Deep Unfolding Networks (DUNs) have made impressive progress in recovering 3D hyperspectral images (HSIs) from a single 2D measurement. However, the inherent nonlinear and ill-posed characteristics of HSI reconstruction still pose challenges to existing methods in terms of accuracy and stability. To address this issue, we propose a Mamba-inspired Joint Unfolding Network (MiJUN), which integrates physics-embedded DUNs with learning-based HSI imaging. Firstly, leveraging the concept of trapezoid discretization to expand the representation space of unfolding networks, we introduce an accelerated unfolding network scheme. This approach can be interpreted as a generalized accelerated half-quadratic splitting with a second-order differential equation, which reduces the reliance on initial optimization stages and addresses challenges related to long-range interactions. Crucially, within the Mamba framework, we restructure the Mamba-inspired global-to-local attention mechanism by incorporating a selective state space model and an attention mechanism. This effectively reinterprets Mamba as a variant of the Transformer} architecture, improving its adaptability and efficiency. Furthermore, we refine the scanning strategy with Mamba by integrating the tensor mode-$k$ unfolding into the Mamba network. This approach emphasizes the low-rank properties of tensors along various modes, while conveniently facilitating 12 scanning directions. Numerical and visual comparisons on both simulation and real datasets demonstrate the superiority of our proposed MiJUN, and achieving overwhelming detail representation.

光谱成像Mamba展开网络高光谱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。