提出稀疏零阶优化方法Meerkat，高效解决非独立同分布客户端的联邦微调难题。
Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
- 仅微调极稀疏且可迁移的参数子集，大幅降低通信开销。
- 高频同步有效缓解数据异质性问题，性能优于全参零阶优化。
- 通过梯度内积轨迹识别极端非独立同分布客户端，支持早停提升模型质量。
联邦学习允许多个分散的非独立同分布（Non-IID）客户端协作微调大语言模型（LLM），但其庞大参数量带来显著内存与通信挑战。本文提出Meerkat，一种针对联邦LLM微调的稀疏零阶优化（ZO）方法。通过仅对一个可迁移、静态且极稀疏的参数子集进行微调，Meerkat实现卓越的通信效率，支持低成本高频同步。理论分析与实验表明，高频通信有效缓解了非独立同分布数据带来的挑战，性能优于全参数零阶优化。此外，实验显示Meerkat在相同通信频率下超越现有稀疏基线。为应对非独立同分布漂移，Meerkat引入可追踪的本地更新，为每个客户端构建虚拟路径。该机制揭示了GradIP现象：服务器维护的预训练梯度与客户端通过零阶估计的梯度内积，在极端非独立同分布客户端上收敛，而在独立同分布客户端上震荡。这一差异行为可用于识别数据高度异质的客户端。基于此信号，提出Meerkat-vp，通过分析GradIP轨迹识别极端非独立同分布客户端，并应用早停策略以提升聚合模型质量。实验验证，Meerkat和Meerkat-vp显著提升了零阶联邦LLM微调的效率与效果。
原文摘要 · Abstract (English)
Federated Learning enables collaborative fine-tuning of Large Language Models (LLMs) across decentralized Non-Independent and Identically Distributed (Non-IID) clients, but such models' massive parameter sizes lead to significant memory and communication challenges. This work introduces Meerkat, a sparse zeroth-order optimization (ZO) method designed for federated LLM fine-tuning. By limiting fine-tuning to a transferable, static, extremely sparse subset of parameters, Meerkat achieves remarkable communication efficiency, enabling cost-effective high-frequency synchronization. With theoretical analysis and experiments, we show that this high-frequency communication effectively mitigates Non-IID data challenges and leads to superior performance compared to full-parameter ZO. Furthermore, experiment results show that Meerkat outperforms existing sparsity baselines with better performance at the same communication frequency. To further handle Non-IID drift, Meerkat leverages traceable local updates and forms a virtual path for each client. This virtual path mechanism reveals the GradIP phenomenon: the inner products between LLM pre-training gradients maintained by server and client gradients estimated via ZO converges for extreme Non-IID clients but oscillates for IID ones. This distinct behavior provides a signal for identifying clients with extreme data heterogeneity. Using this signal, Meerkat-vp is proposed to analyze GradIP trajectories to identify extreme Non-IID clients and applies early stopping to enhance aggregated model quality. Experiments confirm that Meerkat and Meerkat-vp significantly improve the efficiency and effectiveness of ZO federated LLM fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。