arXiv:2609.08115cs.AI2026-09

MoE模型后训练时,保留原始路由结构能显著提升性能。

Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training

论文配图:Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training
图 1 · 摘自论文原文
  • 用软约束保持预训练路由结构,避免强制均匀化
  • 在Moonlight-16B上达45.77分,远超传统方法的31.91
  • 适合需要保留专家协同关系的下游任务

Mixture-of-Experts(MoE)预训练依赖辅助负载均衡损失(LBL)使各专家使用率趋于均匀。后训练阶段则面临不同情况:基础路由已编码非均匀的专家共激活结构,重新施加均匀性目标会将其抹平。我们发现下游性能取决于对这种继承路由结构的软性保留,称为软路由锚定。为此提出路由先验偏置(RPB),一种训练时将路由logits向冻结基础路由的先验值拉近的偏置,同时保持路由可训练。在Moonlight-16B-A3B的数学后训练中,RPB取得45.77的域内准确率,优于重施LBL的31.91和无锚定微调的29.44,且保留更强的域外能力。该优势在第二模型族Qwen3-30B-A3B-Base及独立语料库上复现。基于路由权重、logits或输出分布定义的锚点表现相当,表明效果来自约束的软性而非具体先验。专家共激活图中保留的社区结构与性能增益一致,但硬性强制同一先验会导致性能急剧下降。因此,社区结构是软锚定的结果而非原因。实践启示:后训练应软性保留继承路由结构,因完全平坦化或绝对强制均带来下游代价。代码将于https://github.com/naver-ai/rpb发布。

原文摘要 · Abstract (English)

Mixture-of-Experts (MoE) pretraining relies on an auxiliary load-balancing loss (LBL) to drive per-expert utilization toward uniformity. Post-training inherits a different situation: the base router already encodes non-uniform expert co-activation structure, which a re-imposed uniformity objective flattens away. We show that downstream performance depends instead on holding this inherited routing softly, a principle we term soft router anchoring, and instantiate it as Router Prior Bias (RPB), a training-time bias that pulls the router logits toward a prior read off the frozen base router while leaving the router itself trainable. On math post-training of Moonlight-16B-A3B, RPB attains 45.77 in-domain accuracy against 31.91 under re-applied LBL and 29.44 under unanchored fine-tuning, and retains more out-of-domain capability than either. The ordering against LBL reproduces on a second model family (Qwen3-30B-A3B-Base), and the advantage over LBL is resolvable on an independently sourced corpus. Anchors defined on the router weights, on its logits, or on its output distribution perform comparably with no consistent ordering, which places the effect in the softness of the constraint rather than in the particular prior RPB supplies. Retained community structure in the expert co-activation graph tracks these gains wherever the base router is non-uniform enough for communities to form, yet enforcing the same prior as a hard assignment preserves that structure while performance falls sharply. Community structure is therefore a footprint of soft anchoring rather than its source, and the practical lesson is that inherited routing should be held softly during post-training, since both flattening it toward uniformity and enforcing it absolutely carry a downstream cost. Our code will be released at https://github.com/naver-ai/rpb.

MoE后训练路由优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。