用差分隐私的压缩技巧，让联邦学习更私密、更省通信
Private and Communication-Efficient Federated Learning based on Differentially Private Sketches
- 用计数哈希压缩本地梯度，减少传输量
- 加噪声保护隐私，理论证明收敛性与隐私保障
- 自适应裁剪减轻梯度偏差，适合高敏感数据场景
联邦学习面临参数共享导致的隐私泄露风险和通信效率低两大挑战。为此，我们提出DPSFL方法，利用差分隐私的计数哈希（count sketch）压缩客户端本地梯度，提升通信效率，并对哈希结果添加噪声以确保差分隐私（DP）。本文提供了所提方法在隐私性和收敛性方面的理论分析。在差分隐私学习中，梯度裁剪虽能限制敏感度并控制噪声规模，但会引入梯度偏差，影响模型性能。为缓解此问题，我们进一步提出DPSFL-AC，采用自适应裁剪策略。实验表明，相比现有方法，本方法在隐私保护、通信效率和模型精度方面均表现更优。
原文摘要 · Abstract (English)
Federated learning (FL) faces two primary challenges: the risk of privacy leakage due to parameter sharing and communication inefficiencies. To address these challenges, we propose DPSFL, a federated learning method that utilizes differentially private sketches. DPSFL compresses the local gradients of each client using a count sketch, thereby improving communication efficiency, while adding noise to the sketches to ensure differential privacy (DP). We provide a theoretical analysis of privacy and convergence for the proposed method. Gradient clipping is essential in DP learning to limit sensitivity and constrain the addition of noise. However, clipping introduces bias into the gradients, negatively impacting FL performance. To mitigate the impact of clipping, we propose an enhanced method, DPSFL-AC, which employs an adaptive clipping strategy. Experimental comparisons with existing techniques demonstrate the superiority of our methods concerning privacy preservation, communication efficiency, and model accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。