arXiv:2505.23620stat.MLcs.LG2025-05NeurIPS

针对隐私保护下的分布估计,提出能自适应最优的算法。

Instance-Optimality for Private KL Distribution Estimation

  • 基于局部邻域定义实例最优性,改进隐私估计性能。
  • 在有无隐私约束下均达到常数倍最优,优于传统方法。
  • 结合私有化Good-Turing估计器,适用于真实分布场景。

我们研究从n个独立同分布样本中估计d个符号上的未知离散分布p的难题,目标是最小化真实分布与估计值之间的KL散度。首先构造了极小极大最优的隐私估计器。然而极小极大最优无法反映算法在特定分布p上的表现,且简单极小极大最优的差分隐私估计器在真实分布上可能表现不佳。因此我们从实例最优性视角出发,将算法在分布p上的误差与在p的局部邻域内可达到的最小误差进行比较。在自然的局部邻域定义下,提出了在有无差分隐私约束情况下均达到常数倍实例最优的算法。上界依赖于(私有的)Good-Turing估计器变体,下界采用加法局部邻域,更精确地刻画了在KL散度下分布估计的困难程度,相比先前工作更为严谨。

原文摘要 · Abstract (English)

We study the fundamental problem of estimating an unknown discrete distribution $p$ over $d$ symbols, given $n$ i.i.d. samples from the distribution. We are interested in minimizing the KL divergence between the true distribution and the algorithm's estimate. We first construct minimax optimal private estimators. Minimax optimality however fails to shed light on an algorithm's performance on individual (non-worst-case) instances $p$ and simple minimax-optimal DP estimators can have poor empirical performance on real distributions. We then study this problem from an instance-optimality viewpoint, where the algorithm's error on $p$ is compared to the minimum achievable estimation error over a small local neighborhood of $p$. Under natural notions of local neighborhood, we propose algorithms that achieve instance-optimality up to constant factors, with and without a differential privacy constraint. Our upper bounds rely on (private) variants of the Good-Turing estimator. Our lower bounds use additive local neighborhoods that more precisely captures the hardness of distribution estimation in KL divergence, compared to ones considered in prior works.

分布估计隐私计算实例最优KL散度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。