发现MoE专家比普通神经元更单一,可直接解析成具体语言任务。
The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level

- 用稀疏探针对比专家与密集网络,发现专家更少多义性。
- 越稀疏的路由机制,专家越趋向单一语义,利于解释。
- 自动解析数百个专家,揭示其为细粒度语言操作专家。
Mixture-of-Experts(MoE)架构已成为扩展大语言模型的主流选择,仅对每个词元激活部分参数。尽管其主要优势在于计算效率,但其稀疏性是否使模型比密集前馈网络(FFN)更易解释仍存疑问。我们通过k-稀疏探针对比MoE专家与密集FFN,发现专家神经元始终更少多义性,且随着路由稀疏度增加,差距扩大。这表明稀疏性促使单个神经元及整个专家向单一语义演化。基于此发现,我们从神经元层面转向专家层面作为分析单元。通过自动解析数百个专家,验证了该方法的有效性。结果澄清了专业化争议:专家既非领域专精(如生物),也非简单词元处理器,而是专注于细粒度语言操作或语义任务(如闭合LaTeX括号)。研究表明,MoE在专家层面具有内在可解释性,为大规模模型解释提供了清晰路径。代码已公开:https://github.com/jerryy33/MoE_analysis。
原文摘要 · Abstract (English)
Mixture-of-Experts (MoE) architectures have become the dominant choice for scaling Large Language Models (LLMs), activating only a subset of parameters per token. While MoE architectures are primarily adopted for computational efficiency, it remains an open question whether their sparsity makes them inherently easier to interpret than dense feed-forward networks (FFNs). We compare MoE experts and dense FFNs using $k$-sparse probing and find that expert neurons are consistently less polysemantic, with the gap widening as routing becomes sparser. This suggests that sparsity pressures both individual neurons and entire experts toward monosemanticity. Leveraging this finding, we zoom out from the neuron to the expert level as a more effective unit of analysis. We validate this approach by automatically interpreting hundreds of experts. This analysis allows us to resolve the debate on specialization: experts are neither broad domain specialists (e.g., biology) nor simple token-level processors. Instead, they function as fine-grained task experts, specializing in linguistic operations or semantic tasks (e.g., closing brackets in $\LaTeX{}$). Our findings suggest that MoEs are inherently interpretable at the expert level, providing a clearer path toward large-scale model interpretability. Code is available at: https://github.com/jerryy33/MoE_analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。