提出新指标BeST,快速选出最适合目标任务的预训练模型。
BeST -- A Novel Source Selection Metric for Transfer Learning
- 基于分类任务设计量化优化方法,计算源模型与目标数据相似度。
- 在多个数据集上验证,小样本下仍能准确选出最优源模型。
- 无需训练即可快速筛选,显著降低迁移学习计算开销,适合大规模模型选择。
迁移学习中一个基础但研究较少的问题是如何从大量已训练模型(针对不同源任务)中高效选出对新目标任务表现最佳的模型,尤其在目标数据有限时。本文提出一种新型任务相似性度量方法BeST及其配套策略,能持续有效识别最可迁移的源模型。其核心设计是在分类任务背景下,采用创新的量化级优化过程,通过类早停机制构建不依赖训练的映射近似函数,从而衡量源模型与目标数据间的相似性。该度量可快速计算,在执行耗时的深度神经网络迁移操作前,即能筛选出最佳候选源模型,显著节省计算资源。实验表明,BeST在多种数据集和不同样本数量下均表现优异,具备广泛适用性。
原文摘要 · Abstract (English)
One of the most fundamental, and yet relatively less explored, goals in transfer learning is the efficient means of selecting top candidates from a large number of previously trained models (optimized for various "source" tasks) that would perform the best for a new "target" task with a limited amount of data. In this paper, we undertake this goal by developing a novel task-similarity metric (BeST) and an associated method that consistently performs well in identifying the most transferrable source(s) for a given task. In particular, our design employs an innovative quantization-level optimization procedure in the context of classification tasks that yields a measure of similarity between a source model and the given target data. The procedure uses a concept similar to early stopping (usually implemented to train deep neural networks (DNNs) to ensure generalization) to derive a function that approximates the transfer learning mapping without training. The advantage of our metric is that it can be quickly computed to identify the top candidate(s) for a given target task before a computationally intensive transfer operation (typically using DNNs) can be implemented between the selected source and the target task. As such, our metric can provide significant computational savings for transfer learning from a selection of a large number of possible source models. Through extensive experimental evaluations, we establish that our metric performs well over different datasets and varying numbers of data samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。