用少量标签生成伪标签,高效挑选重要数据,降低训练成本。
Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling
- 仅需少量标签,通过半监督学习生成伪标签
- 基于伪标签训练动态识别难样本,选出高质量数据子集
- 适合标签稀缺、数据分布偏移的现实场景
数据集剪枝通过从大规模数据集中选取信息量高的子集,降低深度学习的存储与训练开销。然而,现有剪枝方法大多依赖完全标注数据,在真实场景中受限于标注成本高、未标注数据多的问题。近期无标签剪枝方法虽缓解此问题,但依赖预训练模型特征来估计样本难度,当目标数据分布与预训练分布差异大时,估计不可靠。本文提出 SemiPrune,一种标签高效的剪枝框架,仅需少量随机标注子集,利用半监督学习为未标注数据生成伪标签,使原本需真实标签的监督剪枝方法可无缝应用于伪标签训练池。我们基于伪标签引发的训练动态估计样本难度并选择核心数据集(coreset)。通过直接学习目标数据分布,该方法更准确捕捉分布特性,提供更可靠的难度信号与选集结果。我们在领域特定、图像退化及长尾数据集上验证,性能超越现有无标签与标签高效基线,且在标准基准上表现竞争力。
原文摘要 · Abstract (English)
Dataset pruning reduces the storage and training costs of deep learning by selecting an informative subset from a large dataset. However, most existing pruning methods require fully labeled data, which limits their applicability in realistic settings where unlabeled data are abundant and annotation is costly. Recent label-free pruning methods address this issue, but they rely on features from pretrained models to estimate example difficulty. This dependence can be unreliable when the target dataset differs substantially from the pretraining distribution. We propose SemiPrune, a label-efficient dataset pruning framework, using only a small randomly labeled subset, that uses semi-supervised learning to generate pseudo-labels for unlabeled data, allowing existing supervised pruning methods that require label information to be seamlessly applied to the resulting pseudo-labeled training pool. We then estimate example difficulty from pseudo-label-induced training dynamics and select a coreset. By learning directly from the target dataset, our method better captures the target distribution and provides more reliable signals for difficulty estimation and coreset selection. We validate our approach on domain-specific, image-corrupted, and long-tailed datasets, where it achieves state-of-the-art performance among label-free and label-efficient baselines, while also demonstrating competitive performance on standard benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。