arXiv:2609.04830cs.LG2026-09

分层多阈值随机压缩,显著降低联邦学习通信开销

Communication-Efficient Personalized Federated Learning via Layer-Wise Multi-Threshold Random Sketching

论文配图:Communication-Efficient Personalized Federated Learning via Layer-Wise Multi-Threshold Random Sketching
图 1 · 摘自论文原文
  • 每层独立设多个阈值,适配不同参数分布特性
  • 用低比特粗略描述参数,通信量大幅减少
  • 适合资源受限设备的个性化联邦学习场景

个性化联邦学习(PFL)在分布式设备上协同训练个性化模型,避免原始数据共享。尽管PFL能应对数据异构性,但在带宽受限系统中,交换高维参数仍带来巨大上下行通信开销。现有的一比特方法虽实现极端压缩,但通常采用全局统一阈值,忽视了各层参数分布与量化敏感性的差异。单一阈值仅提供粗粒度二值信息,无法捕捉参数分布的细粒度变化。为此,本文提出基于分层多阈值随机投影的通信高效PFL框架。该方法为每层分配独立的量化阈值集,使压缩表示能自适应层内统计特性,并通过多区间编码实现对参数的更精细低比特表达。所提方法支持双向通信,使用紧凑的低比特投影,相比现有的一比特压缩方法,在通信-精度权衡上表现更优。

原文摘要 · Abstract (English)

Personalized federated learning (PFL) is a promising paradigm for collaborative learning over distributed devices, where edge nodes collaboratively train personalized models without sharing raw data. Although PFL addresses data heterogeneity by learning client-specific models, it still suffers from substantial uplink and downlink communication costs when exchanging high-dimensional parameters in bandwidth-constrained systems. Recent one-bit methods achieve extreme compression, but they usually rely on a single thresholding rule applied to the whole model. This design has two limitations. First, it overlooks layer-wise differences in parameter distributions and quantization sensitivities. Second, a single threshold provides only coarse binary information and cannot capture fine-grained variations in parameter distributions. To address these issues, we propose a communication-efficient PFL framework via layer-wise multi-threshold random sketching. In the proposed method, each layer is assigned its own set of quantization thresholds, so that the compressed representation can adapt to layer-specific statistics while using multiple intervals to provide a finer low-bit description of sketched parameters. The proposed method supports bidirectional communication using compact low-bit sketches and improves the communication-accuracy tradeoff compared with existing one-bit compression approaches.

联邦学习通信压缩个性化建模随机投影

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。