arXiv:2501.10877cs.LG2025-01被引 4

用二阶优化加速联邦学习,同时保证各设备模型公平性。

Distributed Quasi-Newton Method for Fair and Fast Federated Learning

  • 设计新框架DQN-Fed,结合拟牛顿法加速训练
  • 实现线性-二次收敛率,平均准确率更高且各设备表现均衡
  • 适合对公平性要求高的边缘设备协同训练场景

联邦学习(FL)使边缘设备在中心服务器协调下协同训练模型。主流方法为一阶优化,但收敛慢。为此,本文提出分布式拟牛顿联邦学习(DQN-Fed),一种新型二阶联邦学习框架,在保持拟牛顿法快速收敛优势的同时,确保全局更新使所有客户端的本地损失函数下降,并匹配拟牛顿法的下降速率,从而提升公平性。理论证明DQN-Fed具有线性-二次收敛率。实验验证其在多个联邦数据集上优于现有先进公平联邦学习方法,在公平性、平均准确率和收敛速度上均有显著提升。

原文摘要 · Abstract (English)

Federated learning (FL) is a promising technology that enables edge devices/clients to collaboratively and iteratively train a machine learning model under the coordination of a central server. The most common approach to FL is first-order methods, where clients send their local gradients to the server in each iteration. However, these methods often suffer from slow convergence rates. As a remedy, second-order methods, such as quasi-Newton, can be employed in FL to accelerate its convergence. Unfortunately, similarly to the first-order FL methods, the application of second-order methods in FL can lead to unfair models, achieving high average accuracy while performing poorly on certain clients' local datasets. To tackle this issue, in this paper we introduce a novel second-order FL framework, dubbed \textbf{d}istributed \textbf{q}uasi-\textbf{N}ewton \textbf{fed}erated learning (DQN-Fed). This approach seeks to ensure fairness while leveraging the fast convergence properties of quasi-Newton methods in the FL context. Specifically, DQN-Fed helps the server update the global model in such a way that (i) all local loss functions decrease to promote fairness, and (ii) the rate of change in local loss functions aligns with that of the quasi-Newton method. We prove the convergence of DQN-Fed and demonstrate its \textit{linear-quadratic} convergence rate. Moreover, we validate the efficacy of DQN-Fed across a range of federated datasets, showing that it surpasses state-of-the-art fair FL methods in fairness, average accuracy and convergence speed.

联邦学习二阶优化公平性拟牛顿

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。