通过修正分类置信度偏差，提升大模型在少样本场景下的跨域泛化能力。
Technical report on label-informed logit redistribution for better domain generalization in low-shot classification with foundation models
- 引入置信度错配惩罚项，动态调整错误分类时的对数似然分布。
- 在12个视觉数据集上平均降低6.01%的期望校准误差，最优达9.72%。
- 特别适合低样本量下需稳定推理的视觉分类任务，如医疗影像分析。
基于基础模型的下游视觉分类任务中，置信度校准成为现实系统中的新兴挑战。由于多种原因，即使图像-文本对不匹配，CLIP头部的对数几率值仍保持较大。在少样本设定下，难以通过数据空间方法解决此问题。本文提出一种损失函数中嵌入的惩罚项，当微调过程中发生错误分类时，将部分对数似然按相对幅度转移到真实类别，称为置信度错配惩罚（CMP）。在12个视觉数据集和5个领域泛化数据集上的大量实验表明，该方法显著优于当前最优基准。相比提示学习方法，平均降低6.01%的期望校准误差（ECE），最低降幅4.01%，最高达9.72%。
原文摘要 · Abstract (English)
Confidence calibration is an emerging challenge in real-world decision systems based on foundations models when used for downstream vision classification tasks. Due to various reasons exposed, logit scores on the CLIP head remain large irrespective of whether the image-language pairs reconcile. It is difficult to address in data space, given the few-shot regime. We propose a penalty incorporated into loss objective that penalizes incorrect classifications whenever one is made during finetuning, by moving an amount of log-likelihood to the true class commensurate to the relative amplitudes of the two likelihoods. We refer to it as \textit{confidence misalignment penalty (CMP)}. Extensive experiments on $12$ vision datasets and $5$ domain generalization datasets supports the calibration performance of our method against stat-of-the-art. CMP outperforms the benchmarked prompt learning methods, demonstrating average improvement in Expected Calibration Error (ECE) by average $6.01$\%, $4.01$ \% at minimum and $9.72$\% at maximum.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。