从聚合标签中用弱学习器构建强学习器,首次证明了在特定场景下可行且可高效实现。
Weak to Strong Learning from Aggregate Labels
- 提出基于大袋数据上的弱学习器,通过聚合信息提升小袋数据的预测精度
- 在标签比例学习(LLP)中,小袋数据上可实现接近100%的准确率
- 算法具备概率保证,适用于真实与合成数据集,适合弱监督学习场景
在聚合标签学习中,训练数据由包含特征向量的“袋”及对应袋的聚合标签组成,聚合标签通常为实例标签的平均值(标签比例学习,LLP)或逻辑或(多实例学习,MIL)。目标是训练一个实例级预测器,通常通过最大化满足袋的比例(即预测标签与聚合标签一致的袋)来实现。弱学习器在训练袋上的准确率恒小于1,而强学习器的准确率可任意接近1。本文研究如何利用弱学习器在聚合标签数据上获得强学习器,类比于传统监督学习中的提升算法。结果表明,在LLP中,任何准确率低于1的弱分类器都无法实现提升;但在大袋数据上具有小准确率的弱学习器,可用于在小袋上构造强学习器,且可在多项式时间内完成。还提出基于采样的更高效版本,具有概率保证,并在三个真实和两个合成数据集上验证了其有效性。本工作首次理论分析了从聚合标签中弱到强学习的问题,给出了在LLP中实现该目标的算法,同时证明了在LLP和MIL中提升的不可能性。
原文摘要 · Abstract (English)
In learning from aggregate labels, the training data consists of sets or "bags" of feature-vectors (instances) along with an aggregate label for each bag derived from the (usually {0,1}-valued) labels of its instances. In learning from label proportions (LLP), the aggregate label is the average of the bag's instance labels, whereas in multiple instance learning (MIL) it is the OR. The goal is to train an instance-level predictor, typically achieved by fitting a model on the training data, in particular one that maximizes the accuracy which is the fraction of satisfied bags i.e., those on which the predicted labels are consistent with the aggregate label. A weak learner has at a constant accuracy < 1 on the training bags, while a strong learner's accuracy can be arbitrarily close to 1. We study the problem of using a weak learner on such training bags with aggregate labels to obtain a strong learner, analogous to supervised learning for which boosting algorithms are known. Our first result shows the impossibility of boosting in LLP using weak classifiers of any accuracy < 1 by constructing a collection of bags for which such weak learners (for any weight assignment) exist, while not admitting any strong learner. A variant of this construction also rules out boosting in MIL for a non-trivial range of weak learner accuracy. In the LLP setting however, we show that a weak learner (with small accuracy) on large enough bags can in fact be used to obtain a strong learner for small bags, in polynomial time. We also provide more efficient, sampling based variant of our procedure with probabilistic guarantees which are empirically validated on three real and two synthetic datasets. Our work is the first to theoretically study weak to strong learning from aggregate labels, with an algorithm to achieve the same for LLP, while proving the impossibility of boosting for both LLP and MIL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。