arXiv:2506.22736cs.CV2025-06ICCV被引 6

统一处理医学影像模糊与错位问题,实现修复与融合一步完成。

UniFuse: A Unified All-in-One Framework for Multi-Modal Medical Image Fusion Under Diverse Degradations and Misalignments

  • 引入退化感知提示学习,联合优化图像对齐与修复
  • 通过单阶段框架实现恢复、对齐、融合一体化
  • 适合处理真实临床中质量差、不匹配的多模态影像

当前多模态医学图像融合通常假设源图像质量高且像素级对齐,其性能严重依赖此条件,在处理错位或退化的图像时表现下降。为此,我们提出UniFuse,一种通用融合框架。通过嵌入退化感知提示学习模块,UniFuse 能无缝整合输入图像的多方向信息,并将跨模态对齐与恢复任务关联,实现两者在统一框架内的联合优化。此外,设计了全向统一特征表示方案,利用Spatial Mamba编码多方向特征,缓解模态间特征对齐差异。为实现单一框架内同时完成恢复与融合,提出通用特征恢复与融合模块,基于LoRA原理构建自适应LoRA协同网络(ALSN),结合退化类型引导,实现单阶段联合恢复与融合。相比分步方法,UniFuse 将对齐、恢复、融合统一于一个框架。多数据集实验验证了该方法的有效性,显著优于现有方法。

原文摘要 · Abstract (English)

Current multimodal medical image fusion typically assumes that source images are of high quality and perfectly aligned at the pixel level. Its effectiveness heavily relies on these conditions and often deteriorates when handling misaligned or degraded medical images. To address this, we propose UniFuse, a general fusion framework. By embedding a degradation-aware prompt learning module, UniFuse seamlessly integrates multi-directional information from input images and correlates cross-modal alignment with restoration, enabling joint optimization of both tasks within a unified framework. Additionally, we design an Omni Unified Feature Representation scheme, which leverages Spatial Mamba to encode multi-directional features and mitigate modality differences in feature alignment. To enable simultaneous restoration and fusion within an All-in-One configuration, we propose a Universal Feature Restoration & Fusion module, incorporating the Adaptive LoRA Synergistic Network (ALSN) based on LoRA principles. By leveraging ALSN's adaptive feature representation along with degradation-type guidance, we enable joint restoration and fusion within a single-stage framework. Compared to staged approaches, UniFuse unifies alignment, restoration, and fusion within a single framework. Experimental results across multiple datasets demonstrate the method's effectiveness and significant advantages over existing approaches.

医学影像图像融合联合优化退化处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。