通过梯度范数选样+辅助模型训练,大幅降低图像分类标注成本
LPLgrad: Optimizing Active Learning Through Gradient Norm Sample Selection and Auxiliary Model Training
- 用主模型与辅助模型联合训练,提升特征提取能力
- 基于熵的梯度范数选择不确定性高的样本,标注效率更高
- 在极少标注数据下显著提效,适合数据稀缺场景
机器学习模型在众多领域表现优异,但其成功依赖大量标注数据,而标注过程耗时费力。现有主动学习方法常忽略训练阶段与样本查询阶段的信息协同。为此,我们提出一种新方法LPLgrad,通过双阶段设计优化主动学习:(i) 训练阶段联合训练主模型与辅助模型,在已标注数据上最大化学习效率;(ii) 查询阶段通过计算未标注样本熵值的梯度范数来量化模型不确定性,优先选择梯度范数最高的样本进行标注。该方法在多个真实数据集上的实验表明,在极少量标注样本条件下,准确率远超现有先进方法,且训练与查询时间相当,具备实用价值。
原文摘要 · Abstract (English)
Machine learning models are increasingly being utilized across various fields and tasks due to their outstanding performance and strong generalization capabilities. Nonetheless, their success hinges on the availability of large volumes of annotated data, the creation of which is often labor-intensive, time-consuming, and expensive. Many active learning (AL) approaches have been proposed to address these challenges, but they often fail to fully leverage the information from the core phases of AL, such as training on the labeled set and querying new unlabeled samples. To bridge this gap, we propose a novel AL approach, Loss Prediction Loss with Gradient Norm (LPLgrad), designed to quantify model uncertainty effectively and improve the accuracy of image classification tasks. LPLgrad operates in two distinct phases: (i) {\em Training Phase} aims to predict the loss for input features by jointly training a main model and an auxiliary model. Both models are trained on the labeled data to maximize the efficiency of the learning process, an aspect often overlooked in previous AL methods. This dual-model approach enhances the ability to extract complex input features and learn intrinsic patterns from the data effectively; (ii) {\em Querying Phase} that quantifies the uncertainty of the main model to guide sample selection. This is achieved by calculating the gradient norm of the entropy values for samples in the unlabeled dataset. Samples with the highest gradient norms are prioritized for labeling and subsequently added to the labeled set, improving the model's performance with minimal labeling effort. Extensive evaluations on real-world datasets demonstrate that the LPLgrad approach outperforms state-of-the-art methods by order of magnitude in terms of accuracy on a small number of labeled images, yet achieving comparable training and querying times in multiple image classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。