动态调整量化方式,让联邦学习通信更高效。
OLALa: Online Learned Adaptive Lattice Codes for Heterogeneous Federated Learning
- 客户端在线自适应调整量化器,无需共享原始数据。
- 在多种压缩率下性能优于固定量化方法,收敛更快。
- 适合模型差异大、网络不稳定的异构联邦学习场景。
联邦学习(FL)允许在不共享原始数据的前提下跨分布式客户端协作训练,但高维模型更新的传输导致显著通信开销。通过客户端对模型更新进行量化可缓解该问题,其中带有抖动的格量化器因结构简单且能保持收敛性而备受关注。然而,现有基于格的联邦学习方案通常采用固定量化规则,在模型更新分布随用户和训练轮次变化的异构动态环境中表现不佳。本文提出在线学习自适应格码(OLALa),使每个客户端可通过轻量本地计算在线调整其量化器。我们首先推导了非固定格量化下的收敛性保证,表明合理的格码自适应可收紧收敛上界。随后设计了一种在线学习算法,使客户端在整个联邦学习过程中持续优化量化器,仅需交换紧凑的量化参数。数值实验表明,OLALa在不同量化率下均能稳定提升学习性能,优于传统固定码本及非自适应方案。
原文摘要 · Abstract (English)
Federated learning (FL) enables collaborative training across distributed clients without sharing raw data, often at the cost of substantial communication overhead induced by transmitting high-dimensional model updates. This overhead can be alleviated by having the clients quantize their model updates, with dithered lattice quantizers identified as an attractive scheme due to its structural simplicity and convergence-preserving properties. However, existing lattice-based FL schemes typically rely on a fixed quantization rule, which is suboptimal in heterogeneous and dynamic environments where the model updates distribution varies across users and training rounds. In this work, we propose Online Learned Adaptive Lattices (OLALa), a heterogeneous FL framework where each client can adjust its quantizer online using lightweight local computations. We first derive convergence guarantees for FL with non-fixed lattice quantizers and show that proper lattice adaptation can tighten the convergence bound. Then, we design an online learning algorithm that enables clients to tune their quantizers throughout the FL process while exchanging only a compact set of quantization parameters. Numerical experiments demonstrate that OLALa consistently improves learning performance under various quantization rates, outperforming conventional fixed-codebook and non-adaptive schemes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。