从多个数据源学习分布,关键在查询集合的共现结构。
Learning Distributions from Multiple Data Providers
- 通过共现图判断能否学好分布:连通即可点态一致,完全图才可高效学习。
- 最优样本复杂度从近线性到平方级,取决于查询集的结构,最差为 $\widetilde O(n^2/ε^2)$。
- 若能直接查全集,复杂度降至 $Θ(n/ε^2)$,且不可再优化,适合有完整查询能力的场景。
针对异构且重叠的数据提供方,研究受限条件样本下的分布学习问题。目标是学习定义在有限域 $[n]$ 上的未知分布 $p$。学习者可查询固定集合族 $\mathscr{S} \subseteq 2^{[n]}$,每次查询 $S \in \mathscr{S}$ 返回独立的条件样本 $p(\cdot \mid S)$。可学习性由 $\mathscr{S}$ 对应的共现图决定:若两个元素出现在同一查询集中则连边。当共现图在目标支撑上连通时,点态一致性可达;而泛化学习(PAC)要求共现图为完全图。最优样本复杂度范围从近线性到二次型。对任意具有完全共现图的查询族,最优复杂度为 $\widetilde O(n^2/ε^2)$,且该界在最坏情况下紧致。若 $[n]$ 本身可查询,则普通采样可将复杂度降至 $Θ(n/ε^2)$,且此界不可改进。更一般地,若 $\mathscr{S}$ 满足层级可比性结构条件,最优复杂度为近似线性 $\widetilde Θ(n/ε^2)$,两两查询族即为典型例子。此外,多项式速率 $\widetilde Θ(n^α/ε^2)$ 在 $α \in (1,2)$ 之间均可实现。
原文摘要 · Abstract (English)
Motivated by learning from heterogeneous and overlapping data providers, we study a stylized model of distribution learning from restricted conditional samples. The goal is to learn an unknown distribution $p$ on a finite domain $[n]$. The learner is given a fixed family of queryable sets $\mathscr{S} \subseteq 2^{[n]}$, and each query to $S \in \mathscr{S}$ returns an independent sample from the conditional distribution $p(\cdot \mid S)$. Learnability is governed by the co-occurrence graph associated with $\mathscr{S}$: two domain elements are adjacent if they appear together in some queryable set. Pointwise consistency is achievable when this graph is connected on the target support. PAC learning requires more: it is possible when the co-occurrence graph is complete. The optimal sample complexity of PAC learning ranges from nearly linear to quadratic. Every query family with complete co-occurrence graph admits sample complexity $\widetilde O(n^2/ε^2)$, and this bound is tight in the worst case. On the other hand, if $[n]$ is queryable then ordinary sampling improves the bound to $Θ(n/ε^2)$, and this cannot be improved further even if every set is queryable. More generally, we identify hierarchical comparabilityas a sufficient structural condition on $\mathscr S$ under which the optimal complexity is nearly linear, $\widetilde Θ(n/ε^2)$, with pairwise query families as a canonical example. Finally, the full range of polynomial rates between linear and quadratic is attainable: for every $α\in (1,2)$, there exists a query family with optimal PAC rate $\widetilde Θ(n^α/ε^2)$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。