用轻量方法提升视觉语言模型主动学习的采样效率
Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration
- 引入可微分不确定性校准损失，高效选择关键数据
- 在多个数据集上达到先进性能，计算开销极低
- 对比提示学习与LoRA，为高效主动学习提供实证参考
主动学习（AL）通过有选择地采样最富信息量的数据，显著降低标注成本。针对大规模视觉语言模型的参数量大、不确定性估计难的问题，本文提出一种参数高效的不确定性校准方法，设计可微分损失函数以优化采样质量。在多个数据集和视觉骨干网络上的大量实验表明，该方法性能媲美甚至超越复杂的基于特征的采样策略，同时具有极低的计算开销。此外，本文系统比较了提示学习与低秩适配（LoRA）在样本选择中的表现，为高效主动学习提供了详尽的实证分析。
原文摘要 · Abstract (English)
Active Learning (AL) has emerged as a powerful approach for minimizing labeling costs by selectively sampling the most informative data for neural network model development. Effective AL for large-scale vision-language models necessitates addressing challenges in uncertainty estimation and efficient sampling given the vast number of parameters involved. In this work, we introduce a novel parameter-efficient learning methodology that incorporates uncertainty calibration loss within the AL framework. We propose a differentiable loss function that promotes uncertainty calibration for effectively selecting fewer and most informative data samples for fine-tuning. Through extensive experiments across several datasets and vision backbones, we demonstrate that our solution can match and exceed the performance of complex feature-based sampling techniques while being computationally very efficient. Additionally, we investigate the efficacy of Prompt learning versus Low-rank adaptation (LoRA) in sample selection, providing a detailed comparative analysis of these methods in the context of efficient AL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。