arXiv:2506.12994cs.LGcs.CR2025-06NeurIPS被引 3

提出隐私保护双层优化新算法,实现近最优隐私-误差权衡。

Differentially Private Bilevel Optimization: Efficient Algorithms with Near-Optimal Rates

  • 基于对数凹采样改进,设计高效差分隐私机制。
  • 凸场景下隐私误差逼近单层最优率,非凸场景达当前最优近似驻点率。
  • 不依赖内层问题维度,适合高维敏感数据的元学习与超参优化。

双层优化在元学习、超参数优化等具有层次结构的机器学习任务中广泛应用,但常涉及敏感训练数据,引发隐私关切。本文研究差分隐私下的双层优化问题。在外部目标函数为凸的情况下,给出了纯和近似差分隐私下经验风险过剩的全新上下界,这些界几乎紧致,仅比标准单层差分隐私经验风险最小化(ERM)的最优率多出反映嵌套结构复杂度的额外项。还提供了双层随机优化的总体损失界,通过指数机制和正则化指数机制的高效实现,在多项式时间内达到。关键技术贡献是针对函数评估不精确时的对数凹采样新方法与分析,可能具有独立价值。在非凸情形下,提出了具备当前最优率的新算法,可私密地寻找近似驻点,且其边界不依赖内层问题的维度。

原文摘要 · Abstract (English)

Bilevel optimization, in which one optimization problem is nested inside another, underlies many machine learning applications with a hierarchical structure -- such as meta-learning and hyperparameter optimization. Such applications often involve sensitive training data, raising pressing concerns about individual privacy. Motivated by this, we study differentially private bilevel optimization. We first focus on settings where the outer-level objective is convex, and provide novel upper and lower bounds on the excess empirical risk for both pure and approximate differential privacy. These bounds are nearly tight and essentially match the optimal rates for standard single-level differentially private ERM, up to additional terms that capture the intrinsic complexity of the nested bilevel structure. We also provide population loss bounds for bilevel stochastic optimization. The bounds are achieved in polynomial time via efficient implementations of the exponential and regularized exponential mechanisms. A key technical contribution is a new method and analysis of log-concave sampling under inexact function evaluations, which may be of independent interest. In the non-convex setting, we develop novel algorithms with state-of-the-art rates for privately finding approximate stationary points. Notably, our bounds do not depend on the dimension of the inner problem.

双层优化差分隐私元学习优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。