arXiv:2506.07814cs.CV2025-06被引 16

融合Mamba与CNN的专家模型,一键修复雨雪雾霾等复杂退化图像。

M2Restore: Mixture-of-Experts-based Mamba-CNN Fusion Framework for All-in-One Image Restoration

  • 用CLIP引导的专家门控机制,根据退化类型精准选择处理模块。
  • 双流结构结合CNN局部细节与Mamba长程依赖建模,提升恢复质量。
  • 边缘感知动态门控,让计算资源聚焦退化敏感区域,更高效精准。

自然图像常受雨、雪、雾等复合退化影响,干扰下游视觉任务。现有方法在跨场景泛化能力及局部细节与全局依赖的平衡上仍存不足。为此,提出M2Restore:一种基于混合专家(MoE)的Mamba-CNN融合框架,实现高效鲁棒的全功能图像修复。首先,设计CLIP引导的MoE门控机制,融合任务提示与语义先验,并通过跨模态特征校准实现退化类型精准选专家。其次,构建双流架构,整合CNN的局部表征力与Mamba的长程建模效率,协同优化全局语义关系与局部结构保真度。第三,引入边缘感知动态门控,自适应调整计算注意力至退化敏感区域,实现更高效的修复。在多个图像修复基准上的大量实验验证了其在视觉质量和定量指标上的优越性。

原文摘要 · Abstract (English)

Natural images are often degraded by complex, composite degradations such as rain, snow, and haze, which adversely impact downstream vision applications. While existing image restoration efforts have achieved notable success, they are still hindered by two critical challenges: limited generalization across dynamically varying degradation scenarios and a suboptimal balance between preserving local details and modeling global dependencies. To overcome these challenges, we propose M2Restore, a novel Mixture-of-Experts (MoE)-based Mamba-CNN fusion framework for efficient and robust all-in-one image restoration. M2Restore introduces three key contributions: First, to boost the model's generalization across diverse degradation conditions, we exploit a CLIP-guided MoE gating mechanism that fuses task-conditioned prompts with CLIP-derived semantic priors. This mechanism is further refined via cross-modal feature calibration, which enables precise expert selection for various degradation types. Second, to jointly capture global contextual dependencies and fine-grained local details, we design a dual-stream architecture that integrates the localized representational strength of CNNs with the long-range modeling efficiency of Mamba. This integration enables collaborative optimization of global semantic relationships and local structural fidelity, preserving global coherence while enhancing detail restoration. Third, we introduce an edge-aware dynamic gating mechanism that adaptively balances global modeling and local enhancement by reallocating computational attention to degradation-sensitive regions. This targeted focus leads to more efficient and precise restoration. Extensive experiments across multiple image restoration benchmarks validate the superiority of M2Restore in both visual quality and quantitative performance.

图像修复Mamba混合专家CNN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。