提出新算法，兼顾联邦学习中的隐私保护与快速收敛。
Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
- 结合裁剪、动量和误差反馈，提升优化稳定性。
- 在非凸平滑问题中实现最优收敛率与近最优隐私保障。
- 适合对隐私与性能均有高要求的联邦学习场景。
强差分隐私（DP）与优化保证是联邦学习中理想的双重属性。然而现有算法难以同时满足：要么在受限假设下具备最优隐私性（如梯度有界或数据异构性有限），要么拥有良好优化性能但缺乏隐私保障。为填补这一空白，本文提出并分析了一种新方法——Clip21-SGD2M，其基于裁剪、Heavy-Ball动量与误差反馈的创新组合。针对客户端数据任意异构的非凸平滑分布式问题，证明了该方法在保持最优收敛率的同时，实现了近最优（局部）差分隐私邻域。在非凸逻辑回归与神经网络训练上的数值实验表明，给定相同隐私预算时，Clip21-SGD2M在优化性能上显著优于基线方法。
原文摘要 · Abstract (English)
Strong Differential Privacy (DP) and Optimization guarantees are two desirable properties for a method in Federated Learning (FL). However, existing algorithms do not achieve both properties at once: they either have optimal DP guarantees but rely on restrictive assumptions such as bounded gradients/bounded data heterogeneity, or they ensure strong optimization performance but lack DP guarantees. To address this gap in the literature, we propose and analyze a new method called Clip21-SGD2M based on a novel combination of clipping, heavy-ball momentum, and Error Feedback. In particular, for non-convex smooth distributed problems with clients having arbitrarily heterogeneous data, we prove that Clip21-SGD2M has optimal convergence rate and also near optimal (local-)DP neighborhood. Our numerical experiments on non-convex logistic regression and training of neural networks highlight the superiority of Clip21-SGD2M over baselines in terms of the optimization performance for a given DP-budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。