通过相关噪声提升差分隐私下的模型训练效率
Correlated Noise Mechanisms for Differentially Private Learning
- 设计带相关性的噪声机制，利用噪声抵消提升隐私保护
- 在加权前缀和估计中实现更优的隐私-效用平衡
- 适合关注隐私计算与实际部署的机器学习研究者
本专著探讨了用于差分隐私（DP）的关联噪声机制的设计与分析，重点聚焦于通过加权前缀和估计这一核心方法，在人工智能与机器学习模型的私有训练中的应用。传统DP机制在随机梯度下降（SGD）算法每一步注入独立噪声以保护训练数据隐私，而近期研究表明，引入噪声间的（反）相关性可显著改善隐私-效用权衡，通过后续步骤巧妙抵消前期添加的部分噪声。此类关联噪声机制，包括矩阵机制、因子分解机制以及应用于学习算法的DP-FTRL，在实践中已实现全球规模部署，具有重要影响。
原文摘要 · Abstract (English)
This monograph explores the design and analysis of correlated noise mechanisms for differential privacy (DP), focusing on their application to private training of AI and machine learning models via the core primitive of estimation of weighted prefix sums. While typical DP mechanisms inject independent noise into each step of a stochastic gradient (SGD) learning algorithm in order to protect the privacy of the training data, a growing body of recent research demonstrates that introducing (anti-)correlations in the noise can significantly improve privacy-utility trade-offs by carefully canceling out some of the noise added on earlier steps in subsequent steps. Such correlated noise mechanisms, known variously as matrix mechanisms, factorization mechanisms, and DP-Follow-the-Regularized-Leader (DP-FTRL) when applied to learning algorithms, have also been influential in practice, with industrial deployment at a global scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。