arXiv:2411.18466cs.CV2024-11CVPR被引 111

让专家按任务复杂度自动分工,提升图像修复效率与精度

Complexity Experts are Task-Discriminative Learners for Any Image Restoration

  • 设计可变计算量的复杂度专家,动态匹配任务需求
  • 低复杂度优先分配策略使专家精准分工,推理时跳过无关专家
  • 在多个图像修复任务中超越现有方法,适合高效实用场景

近年统一框架的全功能图像修复模型显著提升了处理多种退化问题的能力。然而,特定任务的参数常在其他任务中闲置,使混合专家(MoE)架构成为自然延伸。但传统MoE表现不一:部分专家意外泛化,另一些却在本职任务中表现不佳,阻碍了推理时跳过无关专家以节省计算的优势。我们归因于传统MoE结构僵化统一。为此,提出“复杂度专家”——具有不同计算量和感受野的灵活专家模块。关键挑战在于任务与专家的匹配,因退化复杂度事先未知。因此采用简单策略:优先分配低复杂度任务。出人意料的是,该策略有效引导任务分派至合适复杂度的专家。大量实验验证,该方法可在推理中避开无关专家,同时保持优异性能。所提MoCE-IR模型优于当前最优方法,证实其高效性与实用性。源码与模型公开于:https://eduardzamfir.github.io/moceir/

原文摘要 · Abstract (English)

Recent advancements in all-in-one image restoration models have revolutionized the ability to address diverse degradations through a unified framework. However, parameters tied to specific tasks often remain inactive for other tasks, making mixture-of-experts (MoE) architectures a natural extension. Despite this, MoEs often show inconsistent behavior, with some experts unexpectedly generalizing across tasks while others struggle within their intended scope. This hinders leveraging MoEs' computational benefits by bypassing irrelevant experts during inference. We attribute this undesired behavior to the uniform and rigid architecture of traditional MoEs. To address this, we introduce ``complexity experts" -- flexible expert blocks with varying computational complexity and receptive fields. A key challenge is assigning tasks to each expert, as degradation complexity is unknown in advance. Thus, we execute tasks with a simple bias toward lower complexity. To our surprise, this preference effectively drives task-specific allocation, assigning tasks to experts with the appropriate complexity. Extensive experiments validate our approach, demonstrating the ability to bypass irrelevant experts during inference while maintaining superior performance. The proposed MoCE-IR model outperforms state-of-the-art methods, affirming its efficiency and practical applicability. The source code and models are publicly available at \href{https://eduardzamfir.github.io/moceir/}{\texttt{eduardzamfir.github.io/MoCE-IR/}}

图像修复专家模型自适应计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。