arXiv:2410.02260cs.LG2024-10

每轮仅传两个标量,大幅降低联邦学习通信开销

FedScalar: Federated Learning with Scalar Communication for Bandwidth-Constrained Networks

  • 每个设备只上传两个标量值,与模型大小无关
  • 实验显示比FedAvg和QSGD节省更多时间和能耗
  • 适合带宽受限的边缘设备场景

在带宽受限的联邦学习场景中,客户端向服务器频繁上传高维模型更新是主要瓶颈,常导致标准联邦学习超出实际通信预算。本文提出FedScalar,一种通信高效的联邦学习算法:每轮每个客户端仅上传两个标量值,无论模型维度$d$多大。客户端将本地更新差异编码为与本地生成随机向量的内积,传输所得标量及生成种子,服务器可据此重构无偏梯度估计,无需高维传输。理论上,FedScalar在光滑非凸损失函数下收敛至平稳点的速率为$O(d/ ext{ } ext{√}K)$;采用Rademacher分布的随机向量相比高斯分布能降低聚合方差。数值模拟表明,在带宽受限环境下,该方法显著优于FedAvg和QSGD,提升实际运行时间和能量效率。

原文摘要 · Abstract (English)

In bandwidth-constrained federated learning~(FL) settings, the repeated upload of high-dimensional model updates from agents to a central server constitutes the primary bottleneck, often rendering standard FL infeasible within practical communication budgets. We propose \emph{FedScalar}, a communication-efficient FL algorithm in which each agent uploads only two scalar values per round, regardless of the model dimension~$d$. Each agent encodes its local update difference as an inner product with a locally generated random vector and transmits the resulting scalar together with the generating seed, enabling the server to reconstruct an unbiased gradient estimate without any high-dimensional transmission. We prove that \emph{FedScalar} achieves a convergence rate of $O(d/\sqrt{K})$ to a stationary point for smooth non-convex loss functions, and show that adopting a Rademacher distribution for the random vector reduces the aggregation variance compared to the Gaussian case. Numerical simulations confirm that the dimension-free upload cost translates into significant improvements in wall-clock time and energy efficiency over \emph{FedAvg} and \emph{QSGD} in bandwidth-constrained settings.

联邦学习通信压缩低带宽标量传输

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。