针对专家模型的遗忘难题,提出精准聚焦专家的高效遗忘方法。
SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
- 通过专家归因定位关键知识对应专家,集中实施遗忘。
- 遗忘质量提升5%,模型性能保留率提高35%,仅修改0.06%参数。
- 适用于各类专家模型,解决动态路由导致的过度遗忘问题。
近期大模型遗忘技术在移除不良数据影响的同时保持模型可用性方面取得显著进展。然而,稀疏的混合专家(MoE)大模型——这一重要类别——在遗忘研究中仍被忽视。鉴于MoE模型性能卓越,我们提出:如何有效、高效地对这类模型进行遗忘?初步研究发现,由于MoE模型的动态路由机制,现有遗忘方法会导致过度遗忘、知识失控擦除及性能大幅下降。为此,我们提出一种新型选择性专家遗忘框架(SEUF)。通过专家归因,遗忘仅集中在与特定知识关联最紧密的活跃专家上;同时,引入锚定损失约束路由器,稳定目标专家的激活状态,实现精准可控的遗忘。SEUF兼容多种标准遗忘算法。大量实验表明,在多个基准和模型架构上,相较于标准遗忘方法,SEUF可使遗忘质量提升最高达5%,模型效用提升35%,且仅需更新0.06%的模型参数。
原文摘要 · Abstract (English)
Recent advancements in LLMs unlearning have shown remarkable success in removing unwanted data-model influences while preserving the model's utility for legitimate knowledge. Despite these strides, sparse Mixture-of-Experts (MoE) LLMs--a key subset of the LLM family--have remained unexplored in the context of unlearning. As MoE LLMs are celebrated for their exceptional performance, we ask:How can unlearning be performed effectively and efficiently on MoE LLMs? Our pilot study shows that the dynamic routing nature of MoE LLMs introduces unique challenges, leading to excessive forgetting, uncontrolled knowledge erasure and substantial utility drops when existing unlearning methods are applied. To address this, we propose a novel Selected-Expert Unlearning Framework (SEUF). Through expert attribution, unlearning is concentrated on the most actively engaged experts for the specified knowledge. Concurrently, an anchor loss is applied to the router to stabilize the active state of this targeted expert, ensuring focused and controlled unlearning. SEUF is compatible with various standard unlearning algorithms. Extensive experiments demonstrate that SEUF enhances both forget quality up to 5% and model utility by 35% on MoE LLMs across various benchmarks and LLM architectures (compared to standard unlearning algorithms), while only unlearning 0.06% of the model parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。