利用地理信息提升联邦学习效果,让相近地区模型更智能共享梯度。
SGFusion: Stochastic Geographic Gradient Fusion in Federated Learning
- 按地理区域分组用户数据,每区训练独立模型并动态融合梯度。
- 在6国心率预测数据上,收敛误差有界且各地区性能显著提升。
- 适合跨区域联邦学习场景,尤其适用于移动设备分布不均的系统。
本文提出随机地理梯度融合(SGFusion),一种新型联邦学习训练算法,旨在利用移动用户的位置信息。该方法将移动设备收集的数据映射到地理区域,并为每个区域训练一个联邦学习模型,以更好地适应该区域的用户数据和行为特征。通过马尔可夫链蒙特卡洛采样优化的层次化随机图(HRG)建模区域间的局部数据相关性。每轮训练中,各区域从HRG中采样的少数其他区域融合其梯度,实现基于概率的梯度共享。该过程采用自注意力机制,使“更相似”的区域具有“更高共享概率”和“更大注意力权重”。实验表明,该方法在不增加计算开销的前提下显著提升模型效用。基于覆盖6个国家的心率预测数据集的理论与实证分析显示,使用SGFusion训练的模型收敛时预期误差有界,且在所有国家均显著优于现有方法,同时保持系统可扩展性。
原文摘要 · Abstract (English)
This paper proposes Stochastic Geographic Gradient Fusion (SGFusion), a novel training algorithm to leverage the geographic information of mobile users in Federated Learning (FL). SGFusion maps the data collected by mobile devices onto geographical zones and trains one FL model per zone, which adapts well to the data and behaviors of users in that zone. SGFusion models the local data-based correlation among geographical zones as a hierarchical random graph (HRG) optimized by Markov Chain Monte Carlo sampling. At each training step, every zone fuses its local gradient with gradients derived from a small set of other zones sampled from the HRG. This approach enables knowledge fusion and sharing among geographical zones in a probabilistic and stochastic gradient fusion process with self-attention weights, such that "more similar" zones have "higher probabilities" of sharing gradients with "larger attention weights." SGFusion remarkably improves model utility without introducing undue computational cost. Extensive theoretical and empirical results using a heart-rate prediction dataset collected across 6 countries show that models trained with SGFusion converge with upper-bounded expected errors and significantly improve utility in all countries compared to existing approaches without notable cost in system scalability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。