用曲率控制隐私,弱化梯度假设,提升模型泛化能力
Quadratic Objective Perturbation: Curvature-Based Differential Privacy

- 以随机二次项替代线性扰动,利用曲率实现强凸性
- 在弱梯度假设下仍保证(ε,δ)差分隐私,且支持近似求解
- 适合现代复杂模型,如深度网络,对高维数据更稳健
目标扰动是差分隐私经验风险最小化中的标准方法。传统线性目标扰动(LOP)通过添加随机线性项实现隐私保护,同时依赖确定性二次项确保强凸性和稳定性。但该方法要求损失函数梯度有界,排除了大量现代机器学习模型。本文提出二次目标扰动(QOP),通过随机二次形式扰动目标函数,利用曲率诱导强凸性并保障问题稳定性,从而将敏感性控制从梯度假设转移到扰动项的谱性质。结果表明,在较弱的梯度假设下仍可实现(ε, δ)-差分隐私。我们进一步分析近似求解情形,证明隐私保证依然成立。还给出了经验过失风险的效用界,并与LOP进行理论和数值比较,凸显基于曲率扰动的优势。最后讨论算法实现,证明可通过现代分裂方法高效求解。
原文摘要 · Abstract (English)
Objective perturbation is a standard mechanism in differentially private empirical risk minimization. In particular, Linear Objective Perturbation (LOP) enforces privacy by adding a random linear term, while strong convexity and stability are ensured by an additional deterministic quadratic term. However, this approach requires the strong assumption of bounded gradients of the loss function, which excludes many modern machine learning models. In this work, we introduce Quadratic Objective Perturbation (QOP), which perturbs the objective with a random quadratic form. This perturbation induces strong convexity and enforces stability of the problem through curvature, thereby enabling privacy and allowing sensitivity to be controlled through spectral properties of the perturbation rather than assumptions on the gradients. As a result, we obtain $(\varepsilon, δ)$-differential privacy under weaker \red{gradient} assumptions. Furthermore, we extend the analysis to account for approximate solutions, showing that privacy guarantees are preserved under inexact solves. Additionally, we derive utility guarantees in terms of empirical excess risk, and provide a theoretical and numerical comparison to LOP, highlighting the advantages of curvature-based perturbations. Finally, we discuss algorithmic aspects and show that the resulting problems can be solved efficiently using modern splitting schemes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。