在保证严格隐私的前提下，实现高效通信的去中心化学习算法。
DP-CSGP: Differentially Private Stochastic Gradient Push with Compressed Communication
- 结合压缩通信与差分隐私机制，提升去中心化学习效率。
- 理论证明其模型精度达到无压缩通信方法的最优水平。
- 适合对隐私和通信成本敏感的分布式机器学习场景。
本文提出一种面向有向图的去中心化学习差分隐私压缩梯度推送算法（DP-CSGP）。不同于现有工作，该算法在确保严格差分隐私（DP）的同时，保持高模型效用并实现高效通信。针对一般非凸且光滑的目标函数，我们证明该算法在每个节点上满足(ε, δ)-DP条件下，能达到紧致的效用界：$Øig( √d·\log(1/δ) / (√n·J·ε) ig)$，其中 $J$ 和 $d$ 分别为本地样本数和决策变量维度，该界与精确通信的去中心化方法一致。在基准任务上的大量实验表明，在相同隐私预算下，与现有精确通信的去中心化方法相比，DP-CSGP 在显著降低通信开销的同时，仍能保持相近的模型准确率。
原文摘要 · Abstract (English)
In this paper, we propose a Differentially Private Stochastic Gradient Push with Compressed communication (termed DP-CSGP) for decentralized learning over directed graphs. Different from existing works, the proposed algorithm is designed to maintain high model utility while ensuring both rigorous differential privacy (DP) guarantees and efficient communication. For general non-convex and smooth objective functions, we show that the proposed algorithm achieves a tight utility bound of $\mathcal{O}\left( \sqrt{d\log \left( \frac{1}δ \right)}/(\sqrt{n}Jε) \right)$ ($J$ and $d$ are the number of local samples and the dimension of decision variables, respectively) with $\left(ε, δ\right)$-DP guarantee for each node, matching that of decentralized counterparts with exact communication. Extensive experiments on benchmark tasks show that, under the same privacy budget, DP-CSGP achieves comparable model accuracy with significantly lower communication cost than existing decentralized counterparts with exact communication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。