通过自校准机制提升视觉语言模型在分布外检测中的表现
Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection
- 引入自校准机制，动态调节两类任务的优化权重
- 仅用少量已知类数据即实现显著提升的分布外检测性能
- 兼容多种提示调优方法，适合需要鲁棒性部署的场景
分布外（OOD）检测对于开放世界应用中可靠机器学习模型的部署至关重要。基于CLIP的OOD检测近期通过使用从已知类数据中提取的OOD特征进行提示调优正则化取得了良好效果。然而，由于前景-背景分割不准确，从已知类数据中挖掘的相关上下文可能产生误导性信息，限制了检测性能。本文提出一种新框架——自校准调优（SCT），以解决该问题，仅需少量已知类数据即可实现有效的OOD检测。SCT在原始学习目标的两个组成部分上分别引入调制因子，根据样本预测不确定性自适应地调整两类任务间的优化方向，从而校准OOD正则化的影响力。该方法可与多种基于提示调优的OOD检测方法兼容。通过大量实验与分析验证了SCT的有效性，代码已公开。
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection is crucial for deploying reliable machine learning models in open-world applications. Recent advances in CLIP-based OOD detection have shown promising results via regularizing prompt tuning with OOD features extracted from ID data. However, the irrelevant context mined from ID data can be spurious due to the inaccurate foreground-background decomposition, thus limiting the OOD detection performance. In this work, we propose a novel framework, namely, Self-Calibrated Tuning (SCT), to mitigate this problem for effective OOD detection with only the given few-shot ID data. Specifically, SCT introduces modulating factors respectively on the two components of the original learning objective. It adaptively directs the optimization process between the two tasks during training on data with different prediction uncertainty to calibrate the influence of OOD regularization, which is compatible with many prompt tuning based OOD detection methods. Extensive experiments and analyses have been conducted to characterize and demonstrate the effectiveness of the proposed SCT. The code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。