在高斯分布下学习非齐次半空间,主动学习难有突破,但成员查询可显著提升效率。
Active Learning of General Halfspaces: Label Queries vs Membership Queries
- 主动学习需指数级未标记样本才能优于被动学习
- 成员查询下仅需约 $\tilde{O}(d \cdot \text{polylog}(1/ε))$ 次查询即可逼近最优误差
- 适用于需要高效标注的机器学习场景,如高维数据分类
研究在高斯分布下的通用(非齐次)半空间学习问题,当存在某种查询访问时。在经典的池式主动学习模型中,算法可自适应地对已采样点进行标签查询,我们建立了严格的信息论下界,排除了对被动学习设置的非平凡改进。具体而言,任何主动学习者所需标签复杂度为 $\tildeΩ(d/("log(m)ε))$,其中 $m$ 为未标记样本数。要超越被动学习的标签复杂度 $\tilde{O}(d/ε)$,主动学习必须依赖 $2^{\text{poly}(d)}$ 数量级的未标记样本。另一方面,在成员查询访问下,即使在泛化模型中,也能克服该下界。我们提出一种计算高效的算法,其查询复杂度为 $\tilde{O}(\min\{1/p, 1/ε\} + d \cdot \text{polylog}(1/ε))$,实现 $O(opt)+ε$ 的误差保证,其中 $p \in [0, 1/2]$ 为偏差,$opt$ 为最优半空间的 0-1 损失。由此可得主动学习与成员查询模型间的强分离。整体结果刻画了在高斯边缘下学习通用半空间的复杂性。
原文摘要 · Abstract (English)
We study the problem of learning general (i.e., not necessarily homogeneous) halfspaces under the Gaussian distribution on $R^d$ in the presence of some form of query access. In the classical pool-based active learning model, where the algorithm is allowed to make adaptive label queries to previously sampled points, we establish a strong information-theoretic lower bound ruling out non-trivial improvements over the passive setting. Specifically, we show that any active learner requires label complexity of $\tildeΩ(d/(\log(m)ε))$, where $m$ is the number of unlabeled examples. Specifically, to beat the passive label complexity of $\tilde{O} (d/ε)$, an active learner requires a pool of $2^{poly(d)}$ unlabeled samples. On the positive side, we show that this lower bound can be circumvented with membership query access, even in the agnostic model. Specifically, we give a computationally efficient learner with query complexity of $\tilde{O}(\min\{1/p, 1/ε\} + d\cdot polylog(1/ε))$ achieving error guarantee of $O(opt)+ε$. Here $p \in [0, 1/2]$ is the bias and $opt$ is the 0-1 loss of the optimal halfspace. As a corollary, we obtain a strong separation between the active and membership query models. Taken together, our results characterize the complexity of learning general halfspaces under Gaussian marginals in these models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。