解决联邦学习中客户端与学习者分布不匹配问题,提升模型训练效果。
Beyond Non-IID: Learner--Client Distribution Mismatch in Federated Learning

- 基于代理数据动态评估客户端贡献,实现影响感知的选源策略。
- 在CIFAR-10上实现更快收敛和更高精度,优于静态与动态基线。
- 适合数据异构性强、需高效利用有限本地数据的联邦学习场景。
联邦学习系统在异构客户端群体中日益普及,但现有实践常隐含假设:聚合的客户端数据分布代表学习者的目标分布,或从所有可用客户端学习对学习者均有益。然而,这种假设在现实中往往不成立。传统客户端选择策略忽视此类偏差,而多数多源迁移学习方法需直接访问本地数据,或采用一次性模型/特征聚合。本文首次系统研究并缓解学习者-客户端群体分布错位的影响。针对学习者持有少量代理数据的实际场景,我们发现客户端贡献随训练轮次显著变化,传统技术难以识别多源迁移下的有效来源。为此,提出一种动态、影响感知的客户端选择框架,通过在学习者特定的代理集上使用留一法评估,估计各客户端对学习者优化目标的潜在效用,优先选择最具信息量的知识源,同时控制统计噪声与数据异构性的负面影响。在异构数据划分下的CIFAR-10实验表明,该方法持续优于静态与动态基线,实现更快收敛与更高准确率。
原文摘要 · Abstract (English)
Federated learning systems are increasingly deployed to facilitate collaborative model training across a heterogeneous client population. Existing practice mostly implicitly assumes that the aggregated client data distribution is representative of the learner's target distribution or that learning from all available clients is uniformly beneficial for the learner distribution. However, such an assumption often does not hold in reality. Traditional client selection strategies in FL literature largely overlook such misalignment, while most existing work on multi-source transfer learning either requires direct access to local data or uses one-shot model/feature aggregation. In this paper, we take the initiative to understand and mitigate the impacts of such learner-client population misalignment. In particular, we consider the practical setting where the learner keeps a small proxy dataset. We observe that client contributions vary significantly across training rounds, and traditional technology is insufficient to identify beneficial sources under multi-source transfer diversity. Then, we propose a dynamic, influence-aware client selection framework that estimates each client's potential utility to the learner's optimization objective using proxy influence signals on a learner-specific proxy set. Via using leave-one-out evaluations, we prioritize the most informative sources of knowledge while controlling the negative impacts of statistical noise and data heterogeneity. Experiments on CIFAR-10 under heterogeneous data partitions demonstrate that our approach consistently outperforms static and dynamic baselines, achieving faster convergence and higher accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。