首次对在线局部隐私学习中的梯度下降进行非渐近分析，给出实用调参指导。
Non-Asymptotic Analysis of Online Local Private Learning with SGD
- 构建在线局部隐私模型下的通用优化框架，支持实时参数估计。
- 在有限样本下证明收敛性，揭示步长、维度和隐私预算的影响。
- 理论与实验结合，为实际应用提供可操作的隐私-精度权衡参考。
差分隐私随机梯度下降（DP-SGD）广泛用于机器学习与统计中具有隐私保障的优化问题。然而，针对在线问题和局部差分隐私（LDP）模型的非渐近收敛分析仍不完善。现有非渐近分析主要针对非私有优化方法，无法直接应用于隐私保护优化。本文首次系统开展该领域分析，提出适用于在线LDP模型的通用框架。假设个体敏感信息按序采集，目标是实时估计群体相关静态参数。核心贡献在于在有限样本条件下完成所提估计器的全面非渐近收敛分析，为用户提供了关于步长、参数维度和隐私预算等超参数对收敛速率影响的实用指导。通过严格的数学推导与精心设计的数值实验，验证了估计器在理论与实践中的有效性。
原文摘要 · Abstract (English)
Differentially Private Stochastic Gradient Descent (DP-SGD) has been widely used for solving optimization problems with privacy guarantees in machine learning and statistics. Despite this, a systematic non-asymptotic convergence analysis for DP-SGD, particularly in the context of online problems and local differential privacy (LDP) models, remains largely elusive. Existing non-asymptotic analyses have focused on non-private optimization methods, and hence are not applicable to privacy-preserving optimization problems. This work initiates the analysis to bridge this gap and opens the door to non-asymptotic convergence analysis of private optimization problems. A general framework is investigated for the online LDP model in stochastic optimization problems. We assume that sensitive information from individuals is collected sequentially and aim to estimate, in real-time, a static parameter that pertains to the population of interest. Most importantly, we conduct a comprehensive non-asymptotic convergence analysis of the proposed estimators in finite-sample situations, which gives their users practical guidelines regarding the effect of various hyperparameters, such as step size, parameter dimensions, and privacy budgets, on convergence rates. Our proposed estimators are validated in the theoretical and practical realms by rigorous mathematical derivations and carefully constructed numerical experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。