建模标签缺失的机制,能提升半监督学习效果
Informative missingness and its implications in semi-supervised learning
- 将标签缺失机制纳入统计模型,用期望最大化算法优化
- 在标签稀疏且缺失具信息量时,错误率低于全标注样本
- 适合标签获取成本高、缺失非随机的场景
半监督学习(SSL)利用少量标注数据和大量未标注数据构建分类器,本质上是不完整数据问题。该问题可在有限混合模型的似然框架下建模,并通过期望最大化(EM)算法求解。理想情况下,完全标注样本更优;但当标签缺失机制依赖于观测特征或类别标签时,缺失指示本身也携带有用信息。在中等类别重叠、标注数据稀疏且缺失为信息性的情况下,建模缺失机制所获信息可能超过缺失标签带来的损失,使分类器预期误差小于仅使用全标注样本的情况。这为基于似然的推断与经验性SSL方法提供了统一的统计框架。
原文摘要 · Abstract (English)
Semi-supervised learning (SSL) constructs classifiers using both labelled and unlabelled data. It leverages information from labelled samples, whose acquisition is often costly or labour-intensive, together with unlabelled data to enhance prediction performance. This defines an incomplete-data problem, which statistically can be formulated within the likelihood framework for finite mixture models that can be fitted using the expectation-maximisation (EM) algorithm. Ideally, one would prefer a completely labelled sample, as one would anticipate that a labelled observation provides more information than an unlabelled one. However, when the mechanism governing label absence depends on the observed features or the class labels or both, the missingness indicators themselves contain useful information. In certain situations, the information gained from modelling the missing-label mechanism can even outweigh the loss due to missing labels, yielding a classifier with a smaller expected error than one based on a completely labelled sample analysed. This improvement arises particularly when class overlap is moderate, labelled data are sparse, and the missingness is informative. Modelling such informative missingness thus offers a coherent statistical framework that unifies likelihood-based inference with the behaviour of empirical SSL methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。