提出新算法提升异构数据下联邦学习的通信效率与收敛稳定性。
Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data
- 分离近端更新与通信,客户端本地处理非光滑项,仅传压缩数据。
- 在1%压缩比下仍保持竞争力模型精度,通信量显著降低。
- 适合资源受限的边缘设备,尤其适用于非凸优化场景。
联邦复合优化(FCO)为分布式边缘网络中带结构约束(如稀疏性)的模型训练提供了有前景的框架。然而,在处理非光滑正则项、统计异质性及有偏压缩限制时,同时实现通信效率与收敛鲁棒性仍是重大挑战。为此,我们提出 FedCEF(联邦复合误差反馈),一种专用于非凸 FCO 的新算法。FedCEF 引入解耦近端更新机制,将近端算子与通信分离,使客户端可本地处理非光滑项并传输压缩信息。为缓解激进量化噪声和非IID数据带来的偏差,FedCEF 集成严格误差反馈机制与控制变量。此外,设计了通信高效的预近端下行策略,使客户端无需显式传输即可精确重构全局控制变量。理论上证明,FedCEF 在一般非凸条件下实现次线性收敛至可控残差误差,该误差可通过步长和批次大小调节。大量真实数据集实验表明,即使在极端压缩比(如1%)下,FedCEF 仍保持竞争性模型精度,相比未压缩基线显著减少总通信量。
原文摘要 · Abstract (English)
Federated Composite Optimization (FCO) has emerged as a promising framework for training models with structural constraints (e.g., sparsity) in distributed edge networks. However, simultaneously achieving communication efficiency and convergence robustness remains a significant challenge, particularly when dealing with non-smooth regularizers, statistical heterogeneity, and the restrictions of biased compression. To address these issues, we propose FedCEF (Federated Composite Error Feedback), a novel algorithm tailored for non-convex FCO. FedCEF introduces a decoupled proximal update scheme that separates the proximal operator from communication, enabling clients to handle non-smooth terms locally while transmitting compressed information. To mitigate the noise from aggressive quantization and the bias from non-IID data, FedCEF integrates a rigorous error feedback mechanism with control variates. Furthermore, we design a communication-efficient pre-proximal downlink strategy that allows clients to exactly reconstruct global control variables without explicit transmission. We theoretically establish that FedCEF achieves sublinear convergence to a bounded residual error under general non-convexity, which is controllable via the step size and batch size. Extensive experiments on real datasets validate FedCEF maintains competitive model accuracy even under extreme compression ratios (e.g., 1%), significantly reducing the total communication volume compared to uncompressed baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。