提出隐式建模方法，高效评估视觉大模型迁移能力
Implicit Modeling for Transferability Estimation of Vision Foundation Models
- 通过隐式建模捕捉模型内在迁移性，无需显式训练
- 在多种架构和任务上表现更稳定、准确且高效
- 适合需要快速选型的模型部署场景
迁移性评估可在不进行完整微调的前提下，识别最适合下游任务的预训练模型，从而降低计算成本并推动预训练-微调范式发展。然而，现有方法在面对架构多样、训练策略各异、任务对齐不同的新兴模型时，常难以准确评估其迁移能力。本文提出隐式迁移性建模（ITM）框架，通过隐式建模各模型的内在迁移性，并结合分治变分近似（DVA）策略，高效逼近嵌入空间演化过程。该设计使模型可泛化至更广泛的模型类型与下游任务。在涵盖多种训练方案和模型类型的综合性基准上的大量实验表明，ITM在稳定性、有效性与效率方面均持续优于现有方法。
原文摘要 · Abstract (English)
Transferability estimation identifies the best pre-trained models for downstream tasks without incurring the high computational cost of full fine-tuning. This capability facilitates deployment and advances the pre-training and fine-tuning paradigm. However, existing methods often struggle to accurately assess transferability for emerging pre-trained models with diverse architectures, training strategies, and task alignments. In this work, we propose Implicit Transferability Modeling (ITM), a novel framework that implicitly models each model's intrinsic transferability, coupled with a Divide-and-Conquer Variational Approximation (DVA) strategy to efficiently approximate embedding space evolution. This design enables generalization across a broader range of models and downstream tasks. Extensive experiments on a comprehensive benchmark--spanning extensive training regimes and a wider variety of model types--demonstrate that ITM consistently outperforms existing methods in terms of stability, effectiveness, and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。