arXiv:2410.18391cs.LGcs.CR2024-10NeurIPS被引 4

提出更高效用户级隐私优化算法,显著降低计算开销。

Faster Algorithms for User-Level Private Stochastic Convex Optimization

  • 设计线性时间算法,在温和光滑性假设下实现最优误差
  • 平滑损失下仅需约 (mn)^{9/8} 次梯度计算,达到最优误差
  • 非光滑损失时仅需 n^{11/8} m^{5/4} 次计算,适合大规模场景

研究在用户级差分隐私(DP)约束下的私有随机凸优化(SCO)。在此设定中,有 $n$ 个用户(如手机),每个用户拥有 $m$ 条数据(如短信),需保护每个用户全部数据的隐私。现有用户级 DP SCO 算法在大规模机器学习场景中不实用,原因在于:(i) 对损失函数光滑性参数有严格假设,且要求用户数随参数维度多项式增长;或 (ii) 计算过慢,平滑损失至少需 $(mn)^{3/2}$ 次梯度计算,非光滑损失需 $(mn)^3$ 次。为解决上述限制,我们提出全新用户级 DP 算法,实现当前最优的额外误差与运行时间。首先,在温和光滑性假设下,开发出线性时间算法,实现非平凡线性时间算法中的最优误差。其次,对任意平滑损失,实现约 $(mn)^{9/8}$ 次梯度计算下的最优误差。第三,对非光滑损失,实现 $n^{11/8} m^{5/4}$ 次梯度计算下的最优误差。此外,我们的算法无需用户数随维度多项式增长。

原文摘要 · Abstract (English)

We study private stochastic convex optimization (SCO) under user-level differential privacy (DP) constraints. In this setting, there are $n$ users (e.g., cell phones), each possessing $m$ data items (e.g., text messages), and we need to protect the privacy of each user's entire collection of data items. Existing algorithms for user-level DP SCO are impractical in many large-scale machine learning scenarios because: (i) they make restrictive assumptions on the smoothness parameter of the loss function and require the number of users to grow polynomially with the dimension of the parameter space; or (ii) they are prohibitively slow, requiring at least $(mn)^{3/2}$ gradient computations for smooth losses and $(mn)^3$ computations for non-smooth losses. To address these limitations, we provide novel user-level DP algorithms with state-of-the-art excess risk and runtime guarantees, without stringent assumptions. First, we develop a linear-time algorithm with state-of-the-art excess risk (for a non-trivial linear-time algorithm) under a mild smoothness assumption. Our second algorithm applies to arbitrary smooth losses and achieves optimal excess risk in $\approx (mn)^{9/8}$ gradient computations. Third, for non-smooth loss functions, we obtain optimal excess risk in $n^{11/8} m^{5/4}$ gradient computations. Moreover, our algorithms do not require the number of users to grow polynomially with the dimension.

隐私优化差分隐私凸优化高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。