解决稀疏专家模型知识编辑的路由不稳问题，实现高效精准更新。
MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
- 通过专家空域投影重参数化，保持路由输入不变
- 在多个数据集上达到最优编辑效果且路由分布稳定
- 适合需要高效、稳定更新的大规模稀疏模型用户
知识编辑（KE）可对大语言模型中的事实内容进行精确修改。现有方法主要针对稠密架构设计，难以直接应用于当前主流的稀疏混合专家（MoE）模型。尽管MoE具备高效扩展能力，但直接套用稠密模型编辑器不仅计算开销大，还易引发路由分布偏移，导致结果不稳定。为此，我们提出MoEEdit，首个面向参数修改型知识编辑的路由稳定框架。该方法通过每个专家的空域投影重新参数化专家更新，使路由器输入保持不变，从而抑制路由变化。优化问题以块结构形式构建，并采用块坐标下降（BCD）求解器高效求解。实验表明，MoEEdit在多项指标上达到当前最佳性能，兼具高特异性与路由稳定性，同时显著降低计算与内存开销。该工作为稀疏大模型的可扩展、精准知识编辑奠定了坚实基础，并强调了路由稳定性在干预中的关键作用。
原文摘要 · Abstract (English)
Knowledge editing (KE) enables precise modifications to factual content in large language models (LLMs). Existing KE methods are largely designed for dense architectures, limiting their applicability to the increasingly prevalent sparse Mixture-of-Experts (MoE) models that underpin modern scalable LLMs. Although MoEs offer strong efficiency and capacity scaling, naively adapting dense-model editors is both computationally costly and prone to routing distribution shifts that undermine stability and consistency. To address these challenges, we introduce MoEEdit, the first routing-stable framework for parameter-modifying knowledge editing in MoE LLMs. Our method reparameterizes expert updates via per-expert null-space projections that keep router inputs invariant and thereby suppress routing shifts. The resulting block-structured optimization is solved efficiently with a block coordinate descent (BCD) solver. Experiments show that MoEEdit attains state-of-the-art efficacy and generalization while preserving high specificity and routing stability, with superior compute and memory efficiency. These results establish a robust foundation for scalable, precise knowledge editing in sparse LLMs and underscore the importance of routing-stable interventions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。