arXiv:2603.18254cs.DScs.CC2026-03被引 1

首次实现私有贝叶斯估计的近优误差,揭示计算与统计间的权衡陷阱。

Computation-Utility-Privacy Tradeoffs in Bayesian Estimation

  • 基于低度框架设计高效算法,实现贝叶斯最优误差
  • 在高维场景下,高效算法误差比指数级算法差一个常数因子
  • 提出新约束机制,用于非鲁棒统计量的隐私化处理

贝叶斯方法在数据受限场景中提供强有力的估计框架,并能合理量化和传播不确定性。然而在实际应用中,保护个体数据隐私的需求日益凸显。尽管已有研究尝试通过分析后验分布的内在隐私性或对现成贝叶斯方法进行隐私化改造来实现差分隐私下的贝叶斯估计,但这些方法通常缺乏严格效用保证,尤其在高维情形下。即使是高斯均值估计和线性回归这类典型任务,在未知参数服从高斯先验的最简情况下,也未明确私有算法能达到的最优误差。本文首次提出两类高效算法,可在均方误差上达到(1+o(1))OPT,同时揭示了两任务均存在有趣的计算-统计差距。对于贝叶斯均值估计,证明所提方法的超额风险在低度框架内是高效的最优解,但仍明显劣于指数时间算法;线性回归亦有类似下界。算法借鉴arXiv:2212.05015中的隐私-鲁棒性框架,但需为本质不鲁棒的统计量(如经验均值、OLS估计)设计基于平方和的鲁棒估计器。过程中还引入一种基于短平坦分解的新约束形式,丰富了平方和工具箱。

原文摘要 · Abstract (English)

Bayesian methods lie at the heart of modern data science and provide a powerful scaffolding for estimation in data-constrained settings and principled quantification and propagation of uncertainty. Yet in many real-world use cases where these methods are deployed, there is a natural need to preserve the privacy of the individuals whose data is being scrutinized. While a number of works have attempted to approach the problem of differentially private Bayesian estimation through either reasoning about the inherent privacy of the posterior distribution or privatizing off-the-shelf Bayesian methods, these works generally do not come with rigorous utility guarantees beyond low-dimensional settings. In fact, even for the prototypical tasks of Gaussian mean estimation and linear regression, it was unknown how close one could get to the Bayes-optimal error with a private algorithm, even in the simplest case where the unknown parameter comes from a Gaussian prior. In this work, we give the first efficient algorithms for both of these problems that achieve mean-squared error $(1+o(1))\mathrm{OPT}$ and additionally show that both tasks exhibit an intriguing computational-statistical gap. For Bayesian mean estimation, we prove that the excess risk achieved by our method is optimal among all efficient algorithms within the low-degree framework, yet is provably worse than what is achievable by an exponential-time algorithm. For linear regression, we prove a qualitatively similar lower bound. Our algorithms draw upon the privacy-to-robustness framework of arXiv:2212.05015, but with the curious twist that to achieve private Bayes-optimal estimation, we need to design sum-of-squares-based robust estimators for inherently non-robust objects like the empirical mean and OLS estimator. Along the way we also add to the sum-of-squares toolkit a new kind of constraint based on short-flat decompositions.

贝叶斯估计差分隐私计算-统计权衡低度框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。