提出首个高效可复现的奇偶函数学习算法,突破统计查询模型限制。
Computationally Efficient Replicable Learning of Parities and Applications
- 设计新算法,从向量集中提取覆盖多数向量的线性子空间。
- 在任意分布下实现奇偶函数的高效可复现学习,样本复杂度多项式级。
- 揭示可复现学习与差分隐私间的样本复杂度权衡,适合理论学习者。
我们研究了可复现性(replicability)与其他稳定性概念之间的计算关系。重点关注可复现的PAC学习及其与差分隐私(Dwork et al. [TCC 2006])和统计查询(SQ)模型(Kearns [JACM `98])的联系。统计上已知差分隐私学习与可复现学习等价,且严格强于SQ学习。但计算上,此前所有高效(即多项式时间)的可复现学习算法仅限于可被SQ学习的任务或受限分布,而差分隐私学习则更广泛。本文首次提出一种在任意分布下对可实现奇偶函数进行高效可复现学习的算法,该任务在SQ模型中是困难的,但在差分隐私下是可行的。这一结果为高效可复现学习在一般分布下严格超越高效SQ学习提供了首个证据,其能力更接近于高效差分隐私学习,尽管二者在计算上存在分离。此外,我们利用该奇偶学习器证明:若假设 $RP eq NP$,将可复现性转化为纯差分隐私需导致样本复杂度严格下降。核心构建块是一种新算法,可高效且可复现地从一组向量中输出其线性张成空间中的一个子空间,覆盖其中大多数向量。
原文摘要 · Abstract (English)
We study the computational relationship between replicability (Impagliazzo et al. [STOC `22], Ghazi et al. [NeurIPS `21]) and other stability notions. Specifically, we focus on replicable PAC learning and its connections to differential privacy (Dwork et al. [TCC 2006]) and to the statistical query (SQ) model (Kearns [JACM `98]). Statistically, it was known that differentially private learning and replicable learning are equivalent and strictly more powerful than SQ-learning. Yet, computationally, all previously known efficient (i.e., polynomial-time) replicable learning algorithms were confined to SQ-learnable tasks or restricted distributions, in contrast to differentially private learning. Our main contribution is the first computationally efficient replicable algorithm for realizable learning of parities over arbitrary distributions, a task that is known to be hard in the SQ-model, but possible under differential privacy. This result provides the first evidence that efficient replicable learning over general distributions strictly extends efficient SQ-learning, and is closer in power to efficient differentially private learning, despite computational separations between replicability and privacy. Additionally, we leverage our parity learner to prove that, assuming $RP \neq NP$, converting replicability to pure differential privacy requires a strict loss in sample complexity. Our main building block is a new, efficient, and replicable algorithm that, given a set of vectors, outputs a subspace of their linear span that covers most of them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。