不用标注数据，用预训练模型和无标签数据高效生成元学习任务
Labeled-Data-Free Meta-Learning: Efficient Task Generation Using Pre-trained Models and Unlabeled Data
- 用预训练模型给无标签数据打软标签生成任务，避免复杂的数据反演
- 在少样本分类上准确率提升8.4%至36.4%，速度提升最高达104倍
- 适合数据隐私受限或标注成本高的实际场景
无标注数据的元学习对现实应用至关重要，因标注数据获取常面临成本高或隐私限制。数据无关元学习（DFML）通过利用预训练模型而不需访问训练数据来应对这一挑战。然而，现有方法依赖模型反演生成训练数据，该过程通常困难且计算开销大，因需生成符合原始分布的高维数据。为此，本文提出一种新元学习范式，通过联合使用预训练模型和无标签数据，避免模型反演。方法通过将预训练模型的输出作为无标签数据的软标签来生成元训练任务。由于任务质量参差不齐，引入基于任务置信度与类别分布均衡性的任务加权机制以保障有效元学习。大量实验表明，本方法显著降低计算开销并提升泛化能力，在少样本分类任务中相较顶尖DFML方法实现最高104倍加速，准确率提升8.4%至36.4%。
原文摘要 · Abstract (English)
Meta-learning without labeled data is crucial for real-world applications, where obtaining labeled datasets can be expensive or restricted due to privacy concerns. Data-Free Meta-Learning (DFML) addresses this challenge by leveraging pre-trained models without access to training data. However, existing DFML methods rely on model inversion to generate training data, a process that is generally difficult and computationally expensive due to the need to generate high-dimensional data matching the original distribution. To address this limitation, we propose a novel meta-learning setting that avoids model inversion by jointly leveraging pre-trained models and unlabeled data. Our method generates meta-training tasks by assigning soft labels from pre-trained models to unlabeled data. Since the quality of these tasks can vary, we introduce a task-weighting mechanism based on task confidence and class distribution balance to ensure effective meta-learning. Extensive experiments demonstrate that our approach substantially reduces computational cost and improves generalization, achieving up to 104-fold speedup and 8.4 percent to 36.4 percent improvements in few-shot classification accuracy compared to state-of-the-art DFML methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。