提出RARE框架,让专家模型的可控生成更稳定高效。
RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models

- 将行为扰动投影到路由器矩阵的零空间,消除路由可见成分
- 有害性攻击成功率53.3%,同时保持67.8% MMLU准确率
- 适用于需要精准控制输出的专家模型应用,如安全对齐
表示工程可通过修改中间隐藏状态实现语言模型行为的轻量级控制,但直接应用于混合专家(MoE)模型时存在结构不匹配问题。我们通过实证研究验证了这一失效模式,发现保持清晰路由可显著恢复控制效果,且路由对语义内容更敏感而非行为变化。为此提出RARE框架,该框架在不依赖路由器的前提下进行表示工程:将任意行为扰动投影至路由器矩阵的零空间,消除路由可见成分,并修正传播至下游层的路由漂移。我们在六种异构开源MoE模型上评估五种估计器,在有害性、真实性与事实编辑三种场景中测试。在有害性控制任务中,RARE平均攻击成功率达53.3%,同时保留67.8% MMLU准确率,优于基线方法。其还将TruthfulQA MC1准确率从41.0%提升至58.6%,CounterFact有效性从16.8%提升至96.3%。结果表明,路由一致性是适配表示工程至MoE模型的关键架构考量。
原文摘要 · Abstract (English)
Representation engineering offers a lightweight means of controlling language-model behavior by modifying intermediate hidden states, but its direct application to Mixture-of-Experts (MoE) models introduces a structural mismatch. We first verify this failure mode through a series of empirical studies and find that preserving clean routing substantially recovers steering performance and that routing is more sensitive to semantic content than to behavioral changes under controlled content. Motivated by these findings, we introduce RARE, a router-agnostic representation engineering framework for MoE language models. RARE projects arbitrary behavioral perturbations onto the null space of the router matrix, thereby removing router-visible components, and further corrects routing drift propagated to selected downstream layers. To decide the best perturbation estimator in this framework, we evaluate five estimators on six heterogeneous open-weight MoE models across three steering scenarios: harmfulness, truthfulness, and factual editing. On harmfulness steering, RARE reaches an average attack success rate of 53.3% while retaining 67.8% MMLU accuracy, yielding a stronger aggregate effectiveness--utility trade-off than baselines. It further improves average TruthfulQA MC1 accuracy from 41.0% to 58.6% and CounterFact efficacy from 16.8% to 96.3%. These results support routing consistency as an important architectural consideration for adapting representation engineering to MoE models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。