arXiv:2507.14559cs.CV2025-07CVPR被引 9

通过分析模型输出的逻辑值演变,快速预测微调效果。

LEAD: Exploring Logit Space Evolution for Model Selection

  • 基于输出逻辑值构建微调动态的非线性建模框架。
  • 在10个下游数据集上验证,可单步预测模型迁移性能。
  • 适合快速筛选预训练模型,尤其适用于数据稀缺场景。

预训练-微调范式在视觉任务中取得显著成功,导致可用预训练模型数量激增,高效选择最适合下游任务的模型成为挑战。核心难点在于如何准确预测模型迁移能力,需考虑微调过程中的优化动态。现有方法通常在特征空间中用线性变换建模微调过程,与微调目标不一致,且忽略优化带来的本质非线性。为此,我们提出LEAD,一种基于网络输出逻辑值(logits)的微调对齐方法。该方法构建理论框架,推导出描述向最终逻辑值状态非线性演化的常微分方程(ODE),并设计类别感知分解方法,以捕捉不同类别间演化动态差异,增强实用性。结合贴近优化目标与非线性建模能力,该方法可一步有效弥合优化差距,避免冗长微调过程。在24个监督与自监督预训练模型、10个下游数据集上的实验表明其表现优异,展现出在低数据场景下的广泛适应性。

原文摘要 · Abstract (English)

The remarkable success of pretrain-then-finetune paradigm has led to a proliferation of available pre-trained models for vision tasks. This surge presents a significant challenge in efficiently choosing the most suitable pre-trained models for downstream tasks. The critical aspect of this challenge lies in effectively predicting the model transferability by considering the underlying fine-tuning dynamics. Existing methods often model fine-tuning dynamics in feature space with linear transformations, which do not precisely align with the fine-tuning objective and fail to grasp the essential nonlinearity from optimization. To this end, we present LEAD, a finetuning-aligned approach based on the network output of logits. LEAD proposes a theoretical framework to model the optimization process and derives an ordinary differential equation (ODE) to depict the nonlinear evolution toward the final logit state. Additionally, we design a class-aware decomposition method to consider the varying evolution dynamics across classes and further ensure practical applicability. Integrating the closely aligned optimization objective and nonlinear modeling capabilities derived from the differential equation, our method offers a concise solution to effectively bridge the optimization gap in a single step, bypassing the lengthy fine-tuning process. The comprehensive experiments on 24 supervised and self-supervised pre-trained models across 10 downstream datasets demonstrate impressive performances and showcase its broad adaptability even in low-data scenarios.

模型选择微调逻辑值迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。