提出EAGLE算法,让不同客户端在联邦学习中获得更公平的性能提升。
Loss Gap Parity for Fairness in Heterogeneous Federated Learning
- 通过最小化各客户端间损失差距的差异,实现相对改进的公平性。
- 在异构数据下显著降低客户端间损失差距,提升最差客户端表现。
- 适用于数据分布差异大的场景,适合关注公平性的系统设计者。
尽管客户端参与联邦学习旨在提升其极少观察到的数据上的性能,但他们往往仍以自身利益为中心,期望全局模型在其本地数据上表现良好。为此,我们提出EAGLE,一种新型联邦学习算法,显式正则化全局模型以最小化客户端间的损失差距——即全局模型与仅用本地数据可训练出的最佳模型之间的性能差异。该方法在异构设置下尤为有效,此时各客户端的最优本地模型可能不一致。不同于现有追求损失均等化、可能损害多数客户端性能的方法,EAGLE聚焦于相对改进的公平性。我们在非凸损失函数下提供了EAGLE的理论收敛保证,并利用新的异构性度量刻画其迭代行为相对于标准联邦学习目标的表现。实验表明,EAGLE通过优先优化离本地最优损失最远的客户端,显著降低客户端间损失差距,在凸与非凸情况下均保持与强基线相当的实用性。
原文摘要 · Abstract (English)
While clients may join federated learning to improve performance on data they rarely observe locally, they often remain self-interested, expecting the global model to perform well on their own data. This motivates an objective that ensures all clients achieve a similar loss gap -the difference in performance between the global model and the best model they could train using only their local data-. To this end, we propose EAGLE, a novel federated learning algorithm that explicitly regularizes the global model to minimize disparities in loss gaps across clients. Our approach is particularly effective in heterogeneous settings, where the optimal local models of the clients may be misaligned. Unlike existing methods that encourage loss parity, potentially degrading performance for many clients, EAGLE targets fairness in relative improvements. We provide theoretical convergence guarantees for EAGLE under non-convex loss functions, and characterize how its iterates perform relative to the standard federated learning objective using a novel heterogeneity measure. Empirically, we demonstrate that EAGLE reduces the disparity in loss gaps among clients by prioritizing those furthest from their local optimal loss, while maintaining competitive utility in both convex and non-convex cases compared to strong baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。