用随机打乱消除固定扫描偏见，提升多模态图像融合效果
Shuffle Mamba: State Space Models with Random Shuffle for Multi-Modal Image Fusion
- 引入随机打乱与逆打乱机制，打破固定扫描顺序
- 在多个数据集上实现优于现有方法的融合质量
- 适合需要无偏长程建模的多模态图像处理任务
多模态图像融合通过整合不同模态的互补信息，生成更丰富、更具信息量的图像。尽管状态空间模型（如 Mamba）以线性复杂度擅长长程建模，但多数基于 Mamba 的方法采用固定扫描策略，可能引入有偏先验。为此，我们提出一种受贝叶斯启发的随机打乱扫描策略，并辅以理论上可行的逆打乱操作，以保持信息协调不变性，旨在消除固定序列扫描带来的偏见。基于这一变换对，我们构建了定制化的 Shuffle Mamba 框架，实现了模态感知的信息表示与跨模态的空间及通道维度交互，确保鲁棒的交互和无偏的全局感受野，用于多模态图像融合。此外，我们设计了一种基于蒙特卡洛平均的测试方法，使模型输出更接近预期结果。在多个多模态图像融合任务上的广泛实验表明，所提方法效果显著，融合质量优于现有先进方法。代码已开源：https://github.com/caoke-963/Shuffle-Mamba。
原文摘要 · Abstract (English)
Multi-modal image fusion integrates complementary information from different modalities to produce enhanced and informative images. Although State-Space Models, such as Mamba, are proficient in long-range modeling with linear complexity, most Mamba-based approaches use fixed scanning strategies, which can introduce biased prior information. To mitigate this issue, we propose a novel Bayesian-inspired scanning strategy called Random Shuffle, supplemented by a theoretically feasible inverse shuffle to maintain information coordination invariance, aiming to eliminate biases associated with fixed sequence scanning. Based on this transformation pair, we customized the Shuffle Mamba Framework, penetrating modality-aware information representation and cross-modality information interaction across spatial and channel axes to ensure robust interaction and an unbiased global receptive field for multi-modal image fusion. Furthermore, we develop a testing methodology based on Monte-Carlo averaging to ensure the model's output aligns more closely with expected results. Extensive experiments across multiple multi-modal image fusion tasks demonstrate the effectiveness of our proposed method, yielding excellent fusion quality compared to state-of-the-art alternatives. The code is available at https://github.com/caoke-963/Shuffle-Mamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。