arXiv:2501.00162cs.LGcs.AI2025-01

针对标签分布极端偏移,提出基于类别选择的迁移学习新方法

Class-based Subset Selection for Transfer Learning under Extreme Label Shift

  • 用Wasserstein距离筛选源域类别并加权,减少域间差异
  • 在标签空间完全不重叠时仍优于现有方法
  • 适合标签分布严重偏移场景,如小样本迁移

现有的迁移学习通常采用两阶段流程:先在大规模源域上预训练,再在目标域少量样本上微调。然而,当输出空间发生分布偏移时,该方法性能显著下降。已有工作仅关注源与目标域共有的类别,忽略了非共享类别的潜在信息。本文提出一种新的少样本迁移学习框架(WaSS),通过Wasserstein距离选择并加权源域类别,以最小化源域与目标域之间的分布距离。我们提供了泛化误差分析,证明该方法对应于边界最小化算法。实验表明,在多个数据集和不同标签偏移设置下,该方法均表现优异,包括标签空间完全不重叠的极端情况。

原文摘要 · Abstract (English)

Existing work within transfer learning often follows a two-step process -- pre-training over a large-scale source domain and then finetuning over limited samples from the target domain. Yet, despite its popularity, this methodology has been shown to suffer in the presence of distributional shift -- specifically when the output spaces diverge. Previous work has focused on increasing model performance within this setting by identifying and classifying only the shared output classes between distributions. However, these methods are inherently limited as they ignore classes outside the shared class set, disregarding potential information relevant to the model transfer. This paper proposes a new process for few-shot transfer learning that selects and weighs classes from the source domain to optimize the transfer between domains. More concretely, we use Wasserstein distance to choose a set of source classes and their weights that minimize the distance between the source and target domain. To justify our proposed algorithm, we provide a generalization analysis of the performance of the learned classifier over the target domain and show that our method corresponds to a bound minimization algorithm. We empirically demonstrate the effectiveness of our approach (WaSS) by experimenting on several different datasets and presenting superior performance within various label shift settings, including the extreme case where the label spaces are disjoint.

迁移学习标签偏移少样本域适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。