用专家嫁接实现私有化大模型联邦学习，节省通信量且不丢知识。
FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Grafting
- 只聚合共享参数，通过动态稀疏性选择本地专家组件进行个性化嫁接。
- 在非独立同分布数据上平均性能优于基线，MMLU基准知识保留率高。
- 适合隐私敏感场景下的个性化大模型训练，尤其适合稀疏专家架构。
大型语言模型的联邦指令微调面临客户端间显著的数据异质性挑战，亟需鲁棒的个性化机制。混合专家（MoE）架构因其专家可针对不同数据模式专精，天然适配此需求。然而，MoE固有的稀疏性——仅激活部分专家——给其与联邦学习（FL）的集成带来挑战。传统联邦框架为密集模型设计，会无差别地聚合所有专家参数，不仅破坏了MoE的动态稀疏性，还可能污染预训练专家中存储的世界知识。为此，我们提出FLEx（Federated LLMs with Personalized Experts），一种基于预训练MoE大模型的高效个性化框架。通过仅聚合共享的非专家参数，FLEx大幅降低通信开销，并保护冻结的预训练专家中的世界知识。为实现个性化，我们引入新颖的专家嫁接机制，利用动态稀疏性从预训练专家中选取组件构建客户端专属专家，并与门控机制联合本地微调。该联合训练使模型学会何时使用共享知识，何时启用个性化专家。在多样化的非独立同分布指令微调数据集上的评估表明，FLEx平均性能持续优于联邦基线，且在知识驱动型基准MMLU上展现出强大的知识保留能力。代码已公开于：https://anonymous.4open.science/r/FLEx-8F12。
原文摘要 · Abstract (English)
Federated instruction tuning of large language models (LLMs) is challenged by significant data heterogeneity across clients, demanding robust personalization. The Mixture of Experts (MoE) architecture, where experts can specialize in distinct data patterns, presents a natural architectural solution to this challenge. The inherent sparsity of the MoE architecture, achieved by selectively activating experts, poses a significant challenge to its integration with federated learning (FL). Conventional FL frameworks, designed for dense models, naively aggregate all expert parameters irrespective of their local activation patterns. This naive approach not only undermines MoE's dynamic sparsity but also risks corrupting the world knowledge within pretrained experts. To address this, we propose FLEx (Federated LLMs with Personalized Experts), a novel framework that leverages pretrained MoE-based LLMs for efficient personalization. By aggregating only the shared non-expert parameters, FLEx significantly reduces communication overhead and preserves the world knowledge stored within the frozen pretrained experts. For personalization, we introduce a novel expert grafting mechanism that leverages dynamic sparsity to construct a client-specific expert from selected components of pretrained experts, tailored to local data. This grafted expert is then fine-tuned locally alongside the gating mechanism. This joint training enables the model to learn when to leverage the shared knowledge from frozen experts and when to employ the personalized one. Evaluations on diverse, non-IID instruction tuning datasets show that FLEx consistently outperforms federated baselines on average, while demonstrating strong knowledge preservation on the knowledge-driven benchmark MMLU. Our code is available at \href{https://anonymous.4open.science/r/FLEx-8F12}{\texttt{https://anonymous.4open.science/r/FLEx-8F12}}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。