提出一种高效安全的去中心化联邦学习梯度压缩方法。
Secure Aggregation with Top-K Sparsification in Decentralized Federated Learning

- 用Top-K稀疏化只传重要梯度,降低通信开销。
- 1%稀疏下仍保持与全梯度相当的模型精度。
- 支持节点掉线和合谋攻击下的隐私保护,适合资源受限场景。
安全聚合是缓解联邦学习中梯度泄露的关键技术,但传统方法通信开销随梯度维度线性增长,对大模型尤其不利,且在带宽有限、节点不可靠的去中心化联邦学习中更为显著。顶K梯度稀疏化可通过仅传输部分关键梯度项有效降低通信量,同时保持良好模型精度。然而,各用户选择的顶K项不可预测且不一致,给稀疏安全聚合带来挑战。本文研究了在用户掉线和合谋条件下,基于信息论安全的去中心化联邦学习中的顶K稀疏化安全聚合。提出一种通信高效的稀疏安全聚合方案,将依赖维度的开销转移至离线阶段,并通过随机掩码与置换保护私有梯度。实验表明,该方案在仅使用1%梯度稀疏化时,仍能保持与全梯度聚合相当的模型精度,显著降低通信成本。
原文摘要 · Abstract (English)
Secure aggregation is a vital component for mitigating gradient leakage in federated learning, but its communication cost conventionally scales with the gradient dimension. This becomes prohibitive for large models and even more pronounced in decentralized federated learning with limited bandwidth and unreliable nodes. Top-K gradient sparsification is an effective approach to reduce communication by transmitting only a few entries of the full gradient, while maintaining competitive model accuracy. Nevertheless, the top-K entries selected by each user are unpredictable and vary across users, which poses a challenge for efficient sparse secure aggregation. This paper studies information-theoretic secure aggregation with top-K sparsification in decentralized federated learning under user dropouts and user collusion. We propose a communication-efficient sparse secure aggregation scheme that offloads dimension-dependent overhead to an offline phase and protects private gradients using random masks and permutations. Experimental results demonstrate that our scheme preserves accuracy comparable to full-gradient aggregation even with only 1% gradient sparsification, while substantially reducing the communication cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。