arXiv:2502.06044stat.MLcs.LG2025-02

用局部贝叶斯优化实现私密超参数调优,兼顾效率与隐私保护。

Differentially Private Hyperparameter Tuning using Local Bayesian Optimization

  • 通过高斯过程私密近似梯度,实现局部贝叶斯优化
  • 在中高维空间下收敛至局部最优,误差受隐私预算影响
  • 适合需要保护用户数据隐私的机器学习调参场景

超参数调优是机器学习的关键环节,但当验证数据包含敏感用户信息时,搜索机制可能通过选中的配置泄露隐私。现有差分隐私超参数调优方法多依赖近随机搜索,而以往的差分隐私贝叶斯优化通常为全局方法,随超参数维度增加而性能急剧下降。本文研究基于局部贝叶斯优化的差分隐私超参数调优,针对验证目标仅可通过噪声黑盒评估、且梯度不可用或难以计算的场景。提出DP-GIBO框架,利用高斯过程代理模型私密地近似梯度。在合理条件下,证明了DP-GIBO可收敛至局部最优解,误差受隐私预算制约,维度依赖为多项式而非指数级。实验表明,DP-GIBO在多个任务上实现了可扩展的私密超参数调优,在中高维空间中显著优于非私密随机搜索和全局贝叶斯优化基线。

原文摘要 · Abstract (English)

Hyperparameter tuning is a key component of machine learning procedures, but when validation data contain sensitive user information, search mechanisms can leak private information through the selected configuration. Existing differentially private hyperparameter tuning methods often rely on near-random search, while prior differentially private Bayesian optimization approaches are typically global and, therefore, scale poorly with the hyperparameter dimensionality. We study differentially private hyperparameter tuning using local Bayesian optimization, focusing on settings where the validation objective is available only through noisy black box evaluations and gradients are unavailable or impractical to compute. We introduce DP-GIBO, a differentially private local Bayesian optimization framework that privately approximates gradients using a Gaussian Process surrogate. Under suitable conditions, we prove that DP-GIBO converges to a locally optimal hyperparameter configuration up to a privacy-dependent error, with dimensional dependence that is polynomial rather than exponential.Empirically, we show that DP-GIBO provides scalable private hyperparameter tuning across multiple tasks, substantially outperforming non-private random search and global Bayesian optimization baselines in moderate-to-high-dimensional hyperparameter spaces.

差分隐私超参数调优贝叶斯优化隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。