用广义高斯噪声改进隐私机器学习，验证高斯机制最优性
Beyond Laplace and Gaussian: Exploring the Generalized Gaussian Mechanism for Private Machine Learning
- 引入广义高斯噪声机制，扩展隐私保护的噪声分布空间
- 实验表明在隐私学习中，β=2（即高斯机制）表现最佳或相当
- 适合关注隐私机制设计与优化的研究者参考
差分隐私通过随机化数据分析算法实现，但会带来效用与隐私的权衡。现有许多隐私机制基于拉普拉斯和高斯加噪机制。本文探索广义高斯（GG）机制，其加噪项服从概率密度 $ e^{- rac{|x|}{σ^β}} $，其中 $ β\geq1 $，记为 $ GG_{β,σ}(f,D) $。拉普拉斯和高斯机制分别是 $ β=1 $ 与 $ β=2 $ 的特例。我们证明整个 GG 家族满足差分隐私，并将 PRV 会计工具扩展以支持其隐私损失计算。随后，在 PATE 与 DP-SGD 两个典型隐私学习流程中应用该机制。实验覆盖 $ β\in[1,2] $（DP-SGD）和 $ β\in[1,4] $（PATE）的可计算范围。结果表明，在各自可行域内，$ β=2 $（高斯机制）的表现与或优于其他 $ β $ 值，为高斯机制在隐私学习中的广泛应用提供了依据。
原文摘要 · Abstract (English)
Differential privacy (DP) is obtained by randomizing a data analysis algorithm, which necessarily introduces a tradeoff between its utility and privacy. Many DP mechanisms are built upon one of two underlying tools: Laplace and Gaussian additive noise mechanisms. We expand the search space of algorithms by investigating the Generalized Gaussian (GG) mechanism, which samples the additive noise term $x$ with probability proportional to $e^{-\frac{| x |}σ^β }$ for some $β\geq 1$ (denoted $GG_{β, σ}(f,D)$). The Laplace and Gaussian mechanisms are special cases of GG for $β=1$ and $β=2$, respectively. We prove that the full GG family satisfies differential privacy and extend the PRV accountant to support privacy loss computation for these mechanisms. We then instantiate the GG mechanism in two canonical private learning pipelines, PATE and DP-SGD. Empirically, we explore PATE and DP-SGD with the GG mechanism across the computationally feasible values of $β$: $β\in [1,2]$ for DP-SGD and $β\in [1,4]$ for PATE. For both mechanisms, we find that $β=2$ (Gaussian) performs as well as or better than other values in their computational tractable domains.This provides justification for the widespread adoption of the Gaussian mechanism in DP learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。