用双层专家系统让扩散模型一次搞定所有图像修复问题
MiM-DiT: MoE in MoE with Diffusion Transformers for All-in-One Image Restoration
- 双层专家架构：先分大类，再选子专家，精准匹配不同模糊类型
- 在多个真实场景数据集上超越现有方法，尤其在复杂退化组合下表现突出
- 适合需要统一处理多种图像退化的实际应用，如自动驾驶与医疗影像
全功能图像修复面临挑战，因雾霾、模糊、噪声、低光等不同退化类型对修复策略要求各异，单一模型难以有效应对。本文提出一种融合双层混合专家（MoE）架构与预训练扩散模型的统一修复框架。该框架在两个层级运作：跨专家层（Inter-MoE）自适应组合专家组以应对主要退化类型，跨内专家层（Intra-MoE）进一步选择特定子专家处理每类内部的细微差异。此设计实现跨退化类别的粗粒度适应，同时对同类内变化进行细粒度调制，确保在复杂真实退化场景下的高专业化修复效果。大量实验证明，该方法在多个图像修复任务中优于当前最优方案。
原文摘要 · Abstract (English)
All-in-one image restoration is challenging because different degradation types, such as haze, blur, noise, and low-light, impose diverse requirements on restoration strategies, making it difficult for a single model to handle them effectively. In this paper, we propose a unified image restoration framework that integrates a dual-level Mixture-of-Experts (MoE) architecture with a pretrained diffusion model. The framework operates at two levels: the Inter-MoE layer adaptively combines expert groups to handle major degradation types, while the Intra-MoE layer further selects specialized sub-experts to address fine-grained variations within each type. This design enables the model to achieve coarse-grained adaptation across diverse degradation categories while performing fine-grained modulation for specific intra-class variations, ensuring both high specialization in handling complex, real-world corruptions. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art approaches on multiple image restoration task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。