私有学习与在线学习在列表预测中不再等价,需引入新维度衡量隐私可学习性。
Private List Learnability vs. Online List Learnability
- 提出$ k $-单调维数,作为隐私列表学习的新必要条件
- 证明有限$ k $-Littlestone维数不足以保证隐私学习,但仍是必要条件
- 通过单调函数反例揭示隐私与在线学习的不等价性,适合理论研究者
本文研究差分隐私(DP)与在线学习在多标签列表学习框架下的关系。在该设定中,$k$-列表学习器对实例 $x$ 输出 $k$ 个可能预测,若真实标签不在列表中则产生损失。经典多分类学习中,私有可学习性与在线可学习性等价。然而,本文发现该等价在列表学习中不成立:有限的 $k$-Littlestone 维度(表征在线 $k$-列表学习能力的变体)不再是差分隐私 $k$-列表学习的充分条件。尽管如此,它仍为必要条件。通过构造反例——定义在 $\mathbb{N}$ 上具有 $k+1$ 个标签的单调函数类——表明其在线 $k$-列表可学习,但不可差分隐私 $k$-列表学习。为此,本文引入新的组合维度 $k$-单调维数,作为阈值维度的推广。不同于多分类情形下 $k=1$ 时两者共有限,当 $k>1$ 时,$k$-Littlestone 维数与 $k$-单调维数无此关联。我们证明 $k$-单调维数的有限性也是差分隐私 $k$-列表学习的另一必要条件。是否两者同时有限即可推出隐私可学习性,仍是开放问题。
原文摘要 · Abstract (English)
This work explores the connection between differential privacy (DP) and online learning in the context of PAC list learning. In this setting, a $k$-list learner outputs a list of $k$ potential predictions for an instance $x$ and incurs a loss if the true label of $x$ is not included in the list. A basic result in the multiclass PAC framework with a finite number of labels states that private learnability is equivalent to online learnability [Alon, Livni, Malliaris, and Moran (2019); Bun, Livni, and Moran (2020); Jung, Kim, and Tewari (2020)]. Perhaps surprisingly, we show that this equivalence does not hold in the context of list learning. Specifically, we prove that, unlike in the multiclass setting, a finite $k$-Littlestone dimensio--a variant of the classical Littlestone dimension that characterizes online $k$-list learnability--is not a sufficient condition for DP $k$-list learnability. However, similar to the multiclass case, we prove that it remains a necessary condition. To demonstrate where the equivalence breaks down, we provide an example showing that the class of monotone functions with $k+1$ labels over $\mathbb{N}$ is online $k$-list learnable, but not DP $k$-list learnable. This leads us to introduce a new combinatorial dimension, the \emph{$k$-monotone dimension}, which serves as a generalization of the threshold dimension. Unlike the multiclass setting, where the Littlestone and threshold dimensions are finite together, for $k>1$, the $k$-Littlestone and $k$-monotone dimensions do not exhibit this relationship. We prove that a finite $k$-monotone dimension is another necessary condition for DP $k$-list learnability, alongside finite $k$-Littlestone dimension. Whether the finiteness of both dimensions implies private $k$-list learnability remains an open question.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。