提出一种高效且私密的分布式学习算法，兼顾通信与隐私保护。
Communication-Efficient Distributed Learning with Differential Privacy
- 本地训练减少通信频率，梯度裁剪加噪声保障隐私
- 理论证明算法收敛至最优解附近，误差在可控范围内
- 在相同隐私预算下性能优于现有方法，适合数据敏感场景
针对无向网络上的非凸学习问题，本文设计了一种兼具通信效率与数据隐私保护的算法。通过本地训练降低通信频率，利用梯度裁剪与加性噪声实现差分隐私保护。理论证明该算法可收敛至问题的驻点，偏差在可控范围内。同时，在差分隐私框架下提供严格隐私保障，确保共享模型无法反推参与者训练数据。实验显示，在相同隐私预算下，该算法在分类任务中表现优于现有先进方法。
原文摘要 · Abstract (English)
We address nonconvex learning problems over undirected networks. In particular, we focus on the challenge of designing an algorithm that is both communication-efficient and that guarantees the privacy of the agents' data. The first goal is achieved through a local training approach, which reduces communication frequency. The second goal is achieved by perturbing gradients during local training, specifically through gradient clipping and additive noise. We prove that the resulting algorithm converges to a stationary point of the problem within a bounded distance. Additionally, we provide theoretical privacy guarantees within a differential privacy framework that ensure agents' training data cannot be inferred from the trained model shared over the network. We show the algorithm's superior performance on a classification task under the same privacy budget, compared with state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。