提出加权梯度追踪方法，解决分布式优化中的隐私泄露问题。
A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization
- 用衰减权重因子消除梯度追踪的隐私漏洞。
- 在时变异步步长下仍能精确收敛到最优解。
- 适合需要保护数据隐私的分布式学习场景。
本文研究分布式优化中的隐私保护问题，旨在防止优化过程中代理的私有信息被潜在攻击者获取。梯度追踪作为一种提升收敛速度的先进技术，近年被广泛应用于多数一阶算法中。我们首次揭示了梯度追踪固有的隐私泄露风险。基于此，提出一种加权梯度追踪的分布式隐私保护算法，通过衰减权重因子消除梯度追踪中的隐私泄露。随后，在时变异步步长条件下分析了所提算法的收敛性，证明其在温和假设下可精确收敛至最优解。最后，通过经典分布式估计问题和卷积神经网络的分布式训练验证了算法的有效性。
原文摘要 · Abstract (English)
This paper investigates the privacy-preserving distributed optimization problem, aiming to protect agents' private information from potential attackers during the optimization process. Gradient tracking, an advanced technique for improving the convergence rate in distributed optimization, has been applied to most first-order algorithms in recent years. We first reveal the inherent privacy leakage risk associated with gradient tracking. Building upon this insight, we propose a weighted gradient tracking distributed privacy-preserving algorithm, eliminating the privacy leakage risk in gradient tracking using decaying weight factors. Then, we characterize the convergence of the proposed algorithm under time-varying heterogeneous step sizes. We prove the proposed algorithm converges precisely to the optimal solution under mild assumptions. Finally, numerical simulations validate the algorithm's effectiveness through a classical distributed estimation problem and the distributed training of a convolutional neural network.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。