arXiv:2502.11096cs.AIcs.CL2025-02被引 4

无需训练,动态调整专家网络,让大模型行为精准改变。

Mixture of Tunable Experts -- Behavior Modification of DeepSeek-R1 at Inference Time

  • 通过分析模型响应,定位特定行为的专家并动态开关。
  • 关闭10个关键专家,敏感问题拒绝率降低52%且性能不变。
  • 适合研究模型可解释性与行为可控性的研究人员。

我们提出混合可调专家(MoTE),扩展大语言模型的专家混合架构。无需额外训练,MoTE可在推理时实时实现有意义且聚焦的行为调整。通过一种名为‘功能令牌共振成像’(fTRI)的技术分析 DeepSeek-R1 的数字大脑——该技术受 fMRI 启发,使用能诱发特定行为的提示(如‘{time}{place}发生了什么?’)——我们实证识别出与拒绝响应等行为相关的独特专家。利用 MoTE 可对这些行为进行干预和控制:关闭前10个最相关的拒绝型专家(占 R1 总14,848个路由专家的0.07%),在敏感参考提示上实现52%的拒绝率下降,且未影响 MT-Bench 上的表现。随机关闭专家仅引发小幅行为变化并引入噪声;强制激活专家则导致拒绝率显著上升。该方法在可解释性和可调控性方面与稀疏自编码器(SAEs)类似,但无需大规模训练,因在拥有大量专家的 MoE 架构中,专业化已在预训练阶段自然形成。研究发现,专家混合架构中的重要功能机制至少部分可定位在少数特定专家中,而非分散于全模型权重。专家子群可被调节以触发显著行为变化,为理解大模型内部运作提供了新视角。

原文摘要 · Abstract (English)

We present the Mixture-of-Tunable-Experts (MoTE), a method that extends the Mixture-of-Experts architecture of Large Language Models (LLMs). Without additional training, MoTE enables meaningful and focused behavior changes in LLMs on-the-fly during inference time. By analyzing the digital LLM brain of DeepSeek-R1 using a technique we dub 'functional Token Resonance Imaging' (fTRI) -- inspired by fMRI and using prompts designed to elicit specific behavior (e.g., 'What happened {time}{place}?') -- we empirically identify distinctive experts associated with behaviors like refusal responses. Using MoTE we are able to intervene and control such specific behavior. We switched off the top 10 most refusal-relevant experts (0.07% of R1's 14,848 routed experts), achieving a 52% refusal reduction on sensitive reference prompts without performance degradation on MT-Bench. Random expert deactivation resulted in smaller behavioral shifts with increased noise, whereas forced expert activation led to significantly higher refusal rates. Our approach shares similarities with sparse autoencoders (SAEs) in terms of explainability and steerability. Unlike SAEs, MoTE does not require large training efforts, as within MoEs with a vast number of experts, specialization already emerged naturally during pretraining. Our findings suggest that significant functional mechanisms in Mixture-of-Experts architectures can at least partially be localized in a small number of specific experts, rather than being distributed throughout the model's weights. Expert subgroups can be tuned to trigger significant behavior variations, providing insights into the inner workings of LLMs.

大模型控制专家混合行为调节

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。