arXiv:2510.24711cs.CV2025-10中稿 · ICLR被引 20

提出ProMoE框架,用显式路由提升视觉扩散模型的专家分工效率。

Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance

  • 分两步路由:先按功能分条件/无条件图像块,再基于语义原型精细分配。
  • 在ImageNet上优于现有方法,两种训练目标下均实现性能突破。
  • 适合研究视觉生成、扩散模型与专家混合架构的学者参考。

混合专家(MoE)已成为在保持计算效率的同时扩展模型容量的强大范式。尽管在大语言模型中取得显著成功,现有将MoE应用于扩散变压器(DiTs)的尝试收效甚微。我们归因于语言与视觉标记的根本差异:语言标记语义密集且跨标记差异明显,而视觉标记具有空间冗余和功能异质性,阻碍了视觉领域中专家的专精化。为此,我们提出ProMoE,一种具备双阶段路由器的MoE框架,通过显式路由引导促进专家专精。具体而言,该引导促使路由器根据功能角色,通过条件路由将图像标记划分为条件与无条件集合,并基于可学习原型(基于语义内容)通过原型路由优化条件图像标记的分配。此外,原型路由在潜在空间中实现基于相似性的专家分配,自然支持显式语义引导,我们验证了此类引导对视觉MoE至关重要。在此基础上,我们提出路由对比损失,显式增强原型路由过程,提升专家内一致性与专家间多样性。在ImageNet基准上的大量实验表明,ProMoE在修正流与DDPM训练目标下均超越现有最先进方法。代码已公开于https://github.com/ali-vilab/ProMoE。

原文摘要 · Abstract (English)

Mixture-of-Experts (MoE) has emerged as a powerful paradigm for scaling model capacity while preserving computational efficiency. Despite its notable success in large language models (LLMs), existing attempts to apply MoE to Diffusion Transformers (DiTs) have yielded limited gains. We attribute this gap to fundamental differences between language and visual tokens. Language tokens are semantically dense with pronounced inter-token variation, while visual tokens exhibit spatial redundancy and functional heterogeneity, hindering expert specialization in vision MoE. To this end, we present ProMoE, an MoE framework featuring a two-step router with explicit routing guidance that promotes expert specialization. Specifically, this guidance encourages the router to partition image tokens into conditional and unconditional sets via conditional routing according to their functional roles, and refine the assignments of conditional image tokens through prototypical routing with learnable prototypes based on semantic content. Moreover, the similarity-based expert allocation in latent space enabled by prototypical routing offers a natural mechanism for incorporating explicit semantic guidance, and we validate that such guidance is crucial for vision MoE. Building on this, we propose a routing contrastive loss that explicitly enhances the prototypical routing process, promoting intra-expert coherence and inter-expert diversity. Extensive experiments on ImageNet benchmark demonstrate that ProMoE surpasses state-of-the-art methods under both Rectified Flow and DDPM training objectives. Code is available at https://github.com/ali-vilab/ProMoE.

扩散模型专家混合视觉生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。