通过反向前瞻优化，提升去中心化联邦学习的收敛速度与泛化能力。
OledFL: Unleashing the Potential of Decentralized Federated Learning via Opposite Lookahead Enhancement
- 引入反向前瞻机制，优化每轮通信中客户端的初始化状态。
- 在CIFAR10/100上实现最高5%性能提升和8倍加速。
- 理论证明收敛性与泛化界，适合追求高效隐私计算的研究者。
去中心化联邦学习（DFL）在训练速度、隐私保护和通信开销方面优于中心化联邦学习（CFL），但其泛化能力仍显著落后，主要源于严重的参数不一致性。本文提出反向前瞻增强技术（Ole），构建OledFL，通过优化每轮通信中各客户端的初始化，显著提升模型的泛化与收敛速度。同时，严格推导了非凸场景下的收敛速率，并通过均匀稳定性刻画泛化误差界，揭示了OledFL兼具快速收敛与高泛化能力的理论依据。在采用狄利克雷分布与病态分布的CIFAR10和CIFAR100数据集上的大量实验表明，相比主流的DFedAvg方法，OledFL可实现最高5%的性能提升和8倍的加速。
原文摘要 · Abstract (English)
Decentralized Federated Learning (DFL) surpasses Centralized Federated Learning (CFL) in terms of faster training, privacy preservation, and light communication, making it a promising alternative in the field of federated learning. However, DFL still exhibits significant disparities with CFL in terms of generalization ability such as rarely theoretical understanding and degraded empirical performance due to severe inconsistency. In this paper, we enhance the consistency of DFL by developing an opposite lookahead enhancement technique (Ole), yielding OledFL to optimize the initialization of each client in each communication round, thus significantly improving both the generalization and convergence speed. Moreover, we rigorously establish its convergence rate in non-convex setting and characterize its generalization bound through uniform stability, which provides concrete reasons why OledFL can achieve both the fast convergence speed and high generalization ability. Extensive experiments conducted on the CIFAR10 and CIFAR100 datasets with Dirichlet and Pathological distributions illustrate that our OledFL can achieve up to 5\% performance improvement and 8$\times$ speedup, compared to the most popular DFedAvg optimizer in DFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。