用对数似然比判断视觉提示是否优于线性探测，提速百倍且准确率超91%。
When Does Visual Prompting Outperform Linear Probing for Vision-Language Models? A Likelihood Perspective
- 基于对数似然比分析视觉提示与线性探测的优劣
- 相比全量训练提速100倍，准确率最高达91%
- 适合想高效评估模型适配性的研究人员
将预训练模型适配到新任务时，不同数据集上的表现差异显著。视觉提示作为一种先进的参数高效微调方法，能显著提升分布外任务的性能；而线性探测作为标准迁移学习方法，有时反而表现最佳。本文提出对数似然比（LLR）方法，用于分析视觉提示与线性探测的相对优势。结合资源高效的视觉提示近似，该方法在运行时间上相比全量训练最多减少100倍，同时预测准确率最高可达91%。源代码已公开于 https://github.com/IBM/VP-LLR。
原文摘要 · Abstract (English)
Adapting pre-trained models to new tasks can exhibit varying effectiveness across datasets. Visual prompting, a state-of-the-art parameter-efficient transfer learning method, can significantly improve the performance of out-of-distribution tasks. On the other hand, linear probing, a standard transfer learning method, can sometimes become the best approach. We propose a log-likelihood ratio (LLR) approach to analyze the comparative benefits of visual prompting and linear probing. By employing the LLR score alongside resource-efficient visual prompts approximations, our cost-effective measure attains up to a 100-fold reduction in run time compared to full training, while achieving prediction accuracies up to 91%. The source code is available at https://github.com/IBM/VP-LLR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。