用在线调参提升伪标签质量,让无监督数据更好辅助模型训练
Prediction-Powered Semi-Supervised Learning with Online Power Tuning
- 基于伪标签构建无偏梯度估计器,兼顾有标签与无标签数据
- 在线动态调整权重参数,在真实数据上超越经典方法
- 适合追求高精度的半监督学习场景,尤其在标注数据少时有效
预测驱动推断(PPI)是一种新型统计推断技术,通过在有标签和无标签数据上使用伪标签,构建无偏且方差低的参数估计器。本文将该思想拓展至半监督学习(SSL)中的模型训练,提出一种新的无偏梯度估计器。该方法解决了一个关键问题:无标签数据虽可提升性能,但其效果高度依赖伪标签质量;错误伪标签会引入偏差,导致模型表现不佳。为平衡有标签与伪标签数据的贡献,我们引入插值参数,并结合一维在线学习算法,实现参数的实时优化。在合成与真实数据集上的实验验证了该方法的优势,相比经典半监督学习基线及离线调参的PPI方法,性能显著提升。
原文摘要 · Abstract (English)
Prediction-Powered Inference (PPI) is a recently proposed statistical inference technique for parameter estimation that leverages pseudo-labels on both labeled and unlabeled data to construct an unbiased, low-variance estimator. In this work, we extend its core idea to semi-supervised learning (SSL) for model training, introducing a novel unbiased gradient estimator. This extension addresses a key challenge in SSL: while unlabeled data can improve model performance, its benefit heavily depends on the quality of pseudo-labels. Inaccurate pseudo-labels can introduce bias, leading to suboptimal models.To balance the contributions of labeled and pseudo-labeled data, we utilize an interpolation parameter and tune it on the fly, alongside the model parameters, using a one-dimensional online learning algorithm. We verify the practical advantage of our approach through experiments on both synthetic and real datasets, demonstrating improved performance over classic SSL baselines and PPI methods that tune the interpolation parameter offline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。