解决联邦学习中的特征漂移问题,提升模型公平性与性能
Drift-Aware Federated Learning: A Causal Perspective
- 从因果视角分析全局与本地优化器对特征漂移的影响
- 提出CAFE框架,通过特征与参数校准减少多数类和高频客户端的偏差
- 适用于数据分布不均、通信频率差异大的实际联邦学习场景
联邦学习(FL)在保护数据隐私的同时实现多客户端协作训练,通常比单客户端训练表现更优。然而,通信频率和数据分布等因素会导致特征漂移,影响最优性能。本文从因果视角分析模型更新漂移与全局及本地优化器的关系:全局优化器的影响主要源于部分客户端参与服务器更新的频率,而本地优化器的影响通常与数据分布不平衡相关。为此,提出新型框架Causal Drift-Aware Federated Learning(CAFE)。CAFE利用特征不变成分与分类结果间的因果关系,在训练阶段独立校准本地客户端的样本特征与分类器;在推理阶段,消除因频繁通信客户端偏好导致的全局模型漂移。实验表明,CAFE通过特征校准、参数校准和历史信息融合,有效降低了向多数类及高频通信节点的漂移倾向。
原文摘要 · Abstract (English)
Federated learning (FL) facilitates collaborative model training among multiple clients while preserving data privacy, often resulting in enhanced performance compared to models trained by individual clients. However, factors such as communication frequency and data distribution can contribute to feature drift, hindering the attainment of optimal training performance. This paper examine the relationship between model update drift and global as well as local optimizer from causal perspective. The influence of the global optimizer on feature drift primarily arises from the participation frequency of certain clients in server updates, whereas the effect of the local optimizer is typically associated with imbalanced data distributions.To mitigate this drift, we propose a novel framework termed Causal drift-Aware Federated lEarning (CAFE). CAFE exploits the causal relationship between feature-invariant components and classification outcomes to independently calibrate local client sample features and classifiers during the training phase. In the inference phase, it eliminated the drifts in the global model that favor frequently communicating clients.Experimental results demonstrate that CAFE's integration of feature calibration, parameter calibration, and historical information effectively reduces both drift towards majority classes and tendencies toward frequently communicating nodes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。