arXiv:2608.09208cs.LGcs.AI2026-08

针对分布式联邦学习中各层学习率统一导致的收敛慢问题,提出动态调整每层学习率的方法。

FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning

  • 根据模型差异信号,自适应调节不同网络层的学习率
  • 相比原始方法提速近5倍,通信轮次减少超59%
  • 无需额外通信,适合边缘设备和物联网场景

去中心化智能系统在异构设备与有限协同条件下日益依赖去中心化联邦学习(DFL)。然而,在数据非独立同分布(non-IID)情况下,由于使用统一学习率而忽视了各层优化需求,导致收敛效率低下。基础层需保持网络一致性,而特定层则需适应本地数据特征,引发梯度冲突,性能下降。为此,本文提出FedA2L,通过局部更新强度与网络共识约束,动态调整各层学习率。该方法无缝集成至现有DFL协议,无需额外通信或协调。大量实验表明,FedA2L在多种算法、模型结构与数据集上,相较原生DFL实现最高4.94倍加速,通信轮次减少高达59%。同时对严重数据异构、大规模网络及稀疏拓扑具有鲁棒性,显著降低通信开销,适用于资源受限或大规模分布式学习场景,尤其适合边缘与物联网部署。代码已开源:https://github.com/nclabteam/FedA2L。

原文摘要 · Abstract (English)

Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL algorithms, various model architectures, and datasets demonstrate that FedA2L achieves up to 4.94 times faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at https://github.com/nclabteam/FedA2L.

联邦学习自适应学习率去中心化边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。