针对联邦学习中数据分布变化,提出动态专家混合框架提升适应能力。
Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning
- 基于最大均值差异检测分布偏移,动态生成专用全局模型。
- 在多种漂移场景下,准确率提升5.5-12.9个百分点,适应速度加快22-95%。
- 适合需要持续适应真实动态环境的隐私保护联邦学习系统。
联邦学习(FL)可在不共享原始数据的前提下实现跨分布式客户端的协作训练,但在真实场景中,客户端数据分布随时间动态演化,带来显著挑战。本文解决流式联邦学习环境中协变量与标签漂移的关键问题,非平稳数据分布会降低模型性能,需引入中间层以使FL适应分布变化。我们提出ShiftEx——一种感知漂移的专家混合框架,通过最大均值差异(MMD)检测协变量漂移,动态创建并训练专用全局模型。该框架采用隐式记忆机制实现专家复用,并结合设施选址优化,联合最小化协变量偏差、专家创建成本与标签不平衡。理论分析与全面实验表明,在基准数据集上,相比先进基线方法,准确率提升5.5-12.9个百分点,适应速度加快22-95%。所提方法为非平稳、真实世界条件下的联邦学习系统提供可扩展、隐私保护的中间层解决方案,同时最小化通信与计算开销。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training across decentralized clients without sharing raw data, yet faces significant challenges in real-world settings where client data distributions evolve dynamically over time. This paper tackles the critical problem of covariate and label shifts in streaming FL environments, where non-stationary data distributions degrade model performance and necessitate a middleware layer that adapts FL to distributional shifts. We introduce ShiftEx, a shift-aware mixture of experts framework that dynamically creates and trains specialized global models in response to detected distribution shifts using Maximum Mean Discrepancy for covariate shifts. The framework employs a latent memory mechanism for expert reuse and implements facility location-based optimization to jointly minimize covariate mismatch, expert creation costs, and label imbalance. Through theoretical analysis and comprehensive experiments on benchmark datasets, we demonstrate 5.5-12.9 percentage point accuracy improvements and 22-95 % faster adaptation compared to state-of-the-art FL baselines across diverse shift scenarios. The proposed approach offers a scalable, privacy-preserving middleware solution for FL systems operating in non-stationary, real-world conditions while minimizing communication and computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。