用随机投影实现最优隐私保护核学习,兼顾精度与隐私。
Optimal differentially private kernel learning with random projection
- 基于随机投影的核方法,在再生希尔伯特空间中实现隐私保护。
- 在局部强凸条件下,达到平方损失和光滑凸损失的极小最大风险率。
- 首次获得无维度依赖的隐私线性学习风险界,适合注重隐私效率的研究者。
差分隐私已成为隐私保护学习算法的核心。本文在经验风险最小化(ERM)框架下,针对差分隐私核学习的优化问题提出新方法。基于再生希尔伯特空间中的高斯过程与随机投影,设计了一种新的差分隐私核ERM算法。该方法在局部强凸条件下,对平方损失和Lipschitz光滑凸损失函数均实现了极小最大超额风险率。我们进一步证明,基于随机傅里叶特征映射或ℓ₂正则化的现有方法会产生次优超额风险界。关键理论贡献还包括:首次获得基于目标扰动的私有线性ERM的无维度超额风险界,不依赖噪声梯度机制。同时,对已有私有核ERM算法得到更紧的风险界。实验验证了理论结论,表明随机投影能实现统计高效且最优隐私的核学习。研究为差分隐私算法设计提供了新洞见,凸显维度压缩在平衡隐私与效用中的核心作用。
原文摘要 · Abstract (English)
Differential privacy has become a cornerstone in the development of privacy-preserving learning algorithms. This work addresses optimizing differentially private kernel learning within the empirical risk minimization (ERM) framework. We propose a novel differentially private kernel ERM algorithm based on random projection in the reproducing kernel Hilbert space using Gaussian processes. Our method achieves minimax-optimal excess risk rates for both the squared loss and Lipschitz-smooth convex loss functions under a local strong convexity condition. We further show that existing approaches based on alternative dimension reduction techniques, such as random Fourier feature mappings or $\ell_2$ regularization, yield suboptimal excess risk bounds. Our key theoretical contribution also includes the derivation of dimension-free excess risk bounds for objective perturbation-based private linear ERM, marking the first such result that does not rely on noisy gradient-based mechanisms. Additionally, we obtain sharper excess risk bounds for existing differentially private kernel ERM algorithms. Empirical evaluations support our theoretical claims, demonstrating that random projection enables statistically efficient and optimally private kernel learning. These findings provide new insights into the design of differentially private algorithms and highlight the central role of dimension reduction in balancing privacy and utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。