arXiv:2510.20157cs.LGcs.DC2025-10被引 1

自适应差分隐私的分布式训练方法,提升隐私保护下的模型性能与效率。

ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push

  • 动态调整噪声方差与学习率,实现个性化隐私保护。
  • 引入历史梯度融合策略,缓解早期大噪声导致的收敛变慢。
  • 适用于时变通信拓扑,适合实际分布式场景部署。

差分隐私广泛应用于分布式学习中以保护敏感数据,通过在模型更新中添加噪声实现。然而,现有采用固定方差噪声的方法常导致模型性能下降和训练效率降低。为此,我们提出一种新方法:基于方差缩减随机梯度推送的自适应差分隐私分布式学习(ADP-VRSGP)。该方法通过分阶段衰减策略动态调整噪声方差与学习率,加速训练并提升最终模型性能,同时提供节点级个性化隐私保障。为应对早期迭代中大方差噪声引起的收敛缓慢问题,我们引入渐进式梯度融合策略,利用历史梯度信息。此外,ADP-VRSGP结合去中心化推-求和与聚合技术,特别适用于时变通信拓扑。严格的理论分析表明,该方法在合适学习率下可实现鲁棒收敛,显著提升训练稳定性和速度。实验结果验证了其在多种场景下优于现有基线,有效解决了隐私保护分布式学习中的挑战。

原文摘要 · Abstract (English)

Differential privacy is widely employed in decentralized learning to safeguard sensitive data by introducing noise into model updates. However, existing approaches that use fixed-variance noise often degrade model performance and reduce training efficiency. To address these limitations, we propose a novel approach called decentralized learning with adaptive differential privacy via variance-reduced stochastic gradient push (ADP-VRSGP). This method dynamically adjusts both the noise variance and the learning rate using a stepwise-decaying schedule, which accelerates training and enhances final model performance while providing node-level personalized privacy guarantees. To counteract the slowed convergence caused by large-variance noise in early iterations, we introduce a progressive gradient fusion strategy that leverages historical gradients. Furthermore, ADP-VRSGP incorporates decentralized push-sum and aggregation techniques, making it particularly suitable for time-varying communication topologies. Through rigorous theoretical analysis, we demonstrate that ADP-VRSGP achieves robust convergence with an appropriate learning rate, significantly improving training stability and speed. Experimental results validate that our method outperforms existing baselines across multiple scenarios, highlighting its efficacy in addressing the challenges of privacy-preserving decentralized learning.

分布式学习差分隐私自适应优化梯度融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。