DriftGuard让联邦学习应对设备数据漂移,省时省力还更准。
DriftGuard: Mitigating Asynchronous Data Drift in Federated Learning
- 用专家混合架构分离全局与本地参数,灵活适应不同设备数据变化
- 在多个数据集上减少83%重训练开销,准确率仍超现有方法
- 适合资源受限设备的持续学习场景,尤其适用于数据分布不一致的联邦系统
在真实联邦学习部署中,参与训练的设备上的数据分布会随时间演变,导致异步数据漂移——不同设备在不同时间、向不同方向漂移。缓解此类漂移极具挑战:频繁重训练在资源受限设备上成本过高,而间隔过长则导致漂移设备性能下降。我们提出 DriftGuard,一种高效的联邦持续学习框架,可应对异步数据漂移。该框架采用类混合专家(MoE)架构,将共享参数(捕捉全局可迁移知识)与本地参数(适配组内特定分布)分离。这一设计支持两种互补的重训练策略:(i) 全局重训练,在检测到系统级漂移时更新共享参数;(ii) 组重训练,基于 MoE 激活模式识别设备群组,选择性更新其本地参数,且无需共享原始数据。在多个数据集和模型上的实验表明,DriftGuard 在匹配或超越当前最优准确率的同时,总重训练成本降低高达83%。因此,其单位重训练成本下的准确率最高,相较最强基线提升最多达2.3倍。代码已开源:https://github.com/blessonvar/DriftGuard。
原文摘要 · Abstract (English)
In real-world Federated Learning (FL) deployments, data distributions on devices that participate in training evolve over time. This leads to asynchronous data drift, where different devices shift at different times and toward different distributions. Mitigating such drift is challenging: frequent retraining incurs high computational cost on resource-constrained devices, while infrequent retraining degrades performance on drifting devices. We propose DriftGuard, a federated continual learning framework that efficiently adapts to asynchronous data drift. DriftGuard adopts a Mixture-of-Experts (MoE) inspired architecture that separates shared parameters, which capture globally transferable knowledge, from local parameters that adapt to group-specific distributions. This design enables two complementary retraining strategies: (i) global retraining, which updates the shared parameters when system-wide drift is identified, and (ii) group retraining, which selectively updates local parameters for clusters of devices identified via MoE gating patterns, without sharing raw data. Experiments across multiple datasets and models show that DriftGuard matches or exceeds state-of-the-art accuracy while reducing total retraining cost by up to 83%. As a result, it achieves the highest accuracy per unit retraining cost, improving over the strongest baseline by up to 2.3x. DriftGuard is available for download from https://github.com/blessonvar/DriftGuard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。