提出新型去中心化联邦学习算法，支持异步参与与不同通信频率。
Sporadic Gradient Tracking over Directed Graphs: A Theoretical Perspective on Decentralized Federated Learning
- 按需计算梯度，允许客户端自定义更新频率
- 在有向图上实现收敛，容忍数据异构与资源差异
- 适合资源不均、网络不稳定的真实场景
去中心化联邦学习（DFL）使拥有本地数据的客户端以对等方式协作训练通用模型。本文统一解决两大关键挑战：(i) 梯度追踪技术缓解数据异构性；(ii) 考虑客户端资源可用性的多样性。提出首个适用于一般有向图的 exttt{Spod-GT}算法，支持客户特定的梯度计算频率和异构非对称通信频率。在放宽梯度估计方差与客户梯度多样性的假设下，对方法进行严格收敛分析，即使客户端间歇参与，仍保证共识与最优性。在图像分类数据集上的数值实验表明， exttt{Spod-GT}优于主流梯度追踪基线。
原文摘要 · Abstract (English)
Decentralized Federated Learning (DFL) enables clients with local data to collaborate in a peer-to-peer manner to train a generalized model. In this paper, we unify two branches of work that have separately solved important challenges in DFL: (i) gradient tracking techniques for mitigating data heterogeneity and (ii) accounting for diverse availability of resources across clients. We propose $\textit{Sporadic Gradient Tracking}$ ($\texttt{Spod-GT}$), the first DFL algorithm that incorporates these factors over general directed graphs by allowing (i) client-specific gradient computation frequencies and (ii) heterogeneous and asymmetric communication frequencies. We conduct a rigorous convergence analysis of our methodology with relaxed assumptions on gradient estimation variance and gradient diversity of clients, providing consensus and optimality guarantees for GT over directed graphs despite intermittent client participation. Through numerical experiments on image classification datasets, we demonstrate the efficacy of $\texttt{Spod-GT}$ compared to well-known GT baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。