arXiv:2412.14668cs.LGcs.AI2024-12被引 2

通过前向传播减少通信轮次,实现6G低延迟联邦学习

LoLaFL: Low-Latency Federated Learning via Forward-only Propagation

  • 采用前向传播与线性特征学习,支持逐层传输聚合
  • 相比传统联邦学习,延迟降低87%至97%,精度相当
  • 适合对实时性要求高的边缘计算场景

联邦学习(FL)已成为在保护数据隐私的前提下实现分布式边缘学习的主流范式。然而,基于反向传播训练的深度神经网络传统联邦学习难以满足6G移动网络的低延迟需求,主要源于高维模型参数传输和因训练过程随机性导致的大量通信轮次。为此,本文引入最大编码率缩减原理,学习线性判别特征,并将所得白盒神经网络拓展至联邦学习框架,提出低延迟联邦学习(LoLaFL)新方法,实现逐层传输与聚合,显著减少通信轮次。此外,提出两种非线性聚合方案:第一种基于理论证明,最优参数聚合应呈调和平均特性;第二种则利用特征的低秩结构,传输特征的低秩近似协方差矩阵以进一步降低延迟。理论分析与实验表明,与传统联邦学习相比,这两种方案分别实现超过87%和97%的延迟降低,且保持相近精度。

原文摘要 · Abstract (English)

Federated learning (FL) has emerged as a widely adopted paradigm for enabling edge learning with distributed data while ensuring data privacy. However, the traditional FL with deep neural networks trained via backpropagation can hardly meet the low-latency learning requirements in the sixth generation (6G) mobile networks. This challenge mainly arises from the high-dimensional model parameters to be transmitted and the numerous rounds of communication required for convergence due to the inherent randomness of the training process. To address this issue, we adopt the state-of-the-art principle of maximal coding rate reduction to learn linear discriminative features and extend the resultant white-box neural network into FL, yielding the novel framework of Low-Latency Federated Learning (LoLaFL) via forward-only propagation. LoLaFL enables layer-wise transmissions and aggregation with significantly fewer communication rounds, thereby considerably reducing latency. Additionally, we propose two \emph{nonlinear} aggregation schemes for LoLaFL. The first scheme is based on the proof that the optimal NN parameter aggregation in LoLaFL should be harmonic-mean-like. The second scheme further exploits the low-rank structures of the features and transmits the low-rank-approximated covariance matrices of features to achieve additional latency reduction. Theoretic analysis and experiments are conducted to evaluate the performance of LoLaFL. In comparison with traditional FL, the two nonlinear aggregation schemes for LoLaFL can achieve reductions in latency of over 87\% and 97\%, respectively, while maintaining comparable accuracies.

联邦学习低延迟6G通信优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。