arXiv:2506.03044math.STcs.LG2025-06

加速优化提升隐私与鲁棒估计性能,降低迭代次数并强化统计保障。

On the Benefits of Accelerated Optimization in Robust and Private Estimation

  • 采用定制学习率和梯度下界设计加速Frank-Wolfe方法
  • 结合Nesterov动量与高斯机制,实现更优的隐私-鲁棒权衡
  • 适用于线性模型等场景,适合关注隐私保护的研究者

我们研究了加速梯度方法(基于Frank-Wolfe和投影梯度下降)在隐私保护与重尾鲁棒估计中的优势。针对Frank-Wolfe方法,采用定制学习率及约束集上ℓ₂-范数梯度的统一下界;对投影梯度下降,则使用Nesterov动量变体,并在ℝᵖ上优化目标函数。这些加速策略降低了迭代复杂度,从而为经验风险与总体风险最小化提供了更强的统计保证。分析覆盖三类设置:非随机数据、随机无模型数据以及参数模型(线性回归与广义线性模型)。方法上,通过噪声梯度同时处理隐私与鲁棒性:利用高斯机制与高级组合确保差分隐私,通过几何中位数-均值估计器实现重尾鲁棒性,且该方法改善了协变量维度的依赖关系。最后,将所得收敛率与现有边界对比,识别出达到最优收敛的情形。

原文摘要 · Abstract (English)

We study the advantages of accelerated gradient methods, specifically based on the Frank-Wolfe method and projected gradient descent, for privacy and heavy-tailed robustness. Our approaches are as follows: For the Frank-Wolfe method, our technique is based on a tailored learning rate and a uniform lower bound on the gradient of the $\ell_2$-norm over the constraint set. For accelerating projected gradient descent, we use the popular variant based on Nesterov's momentum, and we optimize our objective over $\mathbb{R}^p$. These accelerations reduce iteration complexity, translating into stronger statistical guarantees for empirical and population risk minimization. Our analysis covers three settings: non-random data, random model-free data, and parametric models (linear regression and generalized linear models). Methodologically, we approach both privacy and robustness based on noisy gradients. We ensure differential privacy via the Gaussian mechanism and advanced composition, and we achieve heavy-tailed robustness using a geometric median-of-means estimator, which also sharpens the dependency on the dimension of the covariates. Finally, we compare our rates to existing bounds and identify scenarios where our methods attain optimal convergence.

优化加速隐私保护鲁棒估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。