arXiv:2511.19480cs.LGcs.AI2025-11被引 1

发现MoE模型可被剪枝专家后低成本微调,提出防御新策略。

Exploiting the Experts: Unauthorized Compression in MoE-LLMs

  • 通过任务归因识别关键专家,结合主动学习微调研究剪枝效果。
  • 剪枝后需针对性重对齐才能保持性能,否则精度显著下降。
  • 提出纠缠训练等防御方法,防止未经授权的模型压缩与复用。

混合专家(MoE)架构因其可扩展性和高效性被广泛用于大语言模型。然而其模块化结构带来新威胁:攻击者可通过剪枝专家并廉价微调剩余部分,绕过许可和安全限制。本文系统研究了任务特定场景下MoE-LLM的可剪枝性。首先构建专家归因框架,识别对特定任务贡献最大的专家子集;随后利用主动学习驱动的微调评估剪枝与重对齐的性能权衡。结果揭示出关键的知识损失-恢复权衡:虽可分离部分专家以维持任务准确率,但缺乏定向重对齐会导致严重性能下降。基于此,提出防御策略,包括纠缠专家训练和选择性微调协议,旨在提升模型在未经授权情况下的压缩与微调难度。本工作首次系统评估了MoE模型的安全专用化问题,揭示其模块化的双重用途价值。

原文摘要 · Abstract (English)

Mixture-of-Experts (MoE) architectures are increasingly adopted in large language models (LLMs) for their scalability and efficiency. However, their modular structure introduces a unique vulnerability: adversaries can attempt to compress or repurpose models by pruning experts and cheaply fine-tuning the remainder, effectively bypassing licensing and security constraints. In this paper, we systematically study the prunability of MoE-LLMs under task-specific usage. We first develop an expert attribution framework that identifies the subset of experts most responsible for a given task, then evaluate the performance trade-offs of pruning and re-aligning these experts using active learning-driven fine-tuning. Our findings reveal a critical knowledge loss--recovery trade-off: while certain experts can be isolated to retain task accuracy, significant degradation occurs without targeted re-alignment. Based on this analysis, we propose defense strategies that aim to make MoE models harder to compress and fine-tune without authorization, including entangled expert training and selective fine-tuning protocols that resist unauthorized adaptation. By positioning expert pruning as both a threat vector and a defense target, this work highlights the dual-use nature of MoE modularity and provides the first systematic evaluation framework for secure specialization of MoE-LLMs.

MoE模型模型压缩安全防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。