用1比特压缩通信,实现个性化联邦学习的高效协作。
Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching
- 客户端传输1比特随机投影数据,大幅降低通信开销。
- 在多个数据集上实现与先进算法相当的性能,通信量减少超90%。
- 适合数据异构性强、带宽受限的边缘设备场景。
联邦学习(FL)可在去中心化数据上实现协同训练,但面临双向通信开销大和客户端数据异构性两大挑战。为降低通信成本并适应数据异构,我们提出pFed1BS,一种基于1比特随机投影的个性化联邦学习框架。在个性化FL中,目标从训练单一全局模型转向为每个客户端生成定制模型。本框架中,客户端发送高度压缩的1比特草图,服务器聚合并广播全局1比特共识。为实现有效个性化,引入基于符号的正则项,引导本地模型与全局共识对齐,同时保留本地数据特征。为减轻随机投影的计算负担,采用快速哈达玛变换实现高效投影。理论分析表明,该算法收敛至全局势函数的稳定邻域。数值模拟显示,pFed1BS显著降低通信成本,在多个数据集上表现优于现有通信高效联邦学习算法。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative training across decentralized data, but faces key challenges of bidirectional communication overhead and client-side data heterogeneity. To address communication costs while embracing data heterogeneity, we propose pFed1BS, a novel personalized federated learning framework that achieves extreme communication compression through one-bit random sketching. In personalized FL, the goal shifts from training a single global model to creating tailored models for each client. In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus. To enable effective personalization, we introduce a sign-based regularizer that guides local models to align with the global consensus while preserving local data characteristics. To mitigate the computational burden of random sketching, we employ the Fast Hadamard Transform for efficient projection. Theoretical analysis guarantees that our algorithm converges to a stationary neighborhood of the global potential function. Numerical simulations demonstrate that pFed1BS substantially reduces communication costs while achieving competitive performance compared to advanced communication-efficient FL algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。