用高斯过程增强CLIP，实现小样本下精准的异常检测。
GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection
- 在冻结的CLIP特征上构建类别的单类高斯过程，融合图像与文本不确定性。
- 仅需少量样本（K-shot）和轻量级参数，内存开销为O(CK²)。
- 无需微调，显著提升小样本分类与分布外数据检测性能，适合低资源场景。
我们提出GP-Adapter，一种无需训练的框架，通过高斯过程（GP）不确定性建模增强CLIP（对比语言-图像预训练）在少样本分类和分布外（OOD）检测中的能力。尽管CLIP具备强大的零样本识别性能，但其输出为确定性相似度分数，缺乏关键的不确定性信息，尤其在分布偏移和数据稀缺时表现不足。GP-Adapter在冻结的CLIP嵌入基础上，分别对图像特征使用径向基函数（RBF）核、对文本提示使用线性核，构建模态特异、类别独立的单类高斯过程，并融合其预测统计量，生成具有方差感知的置信度分数用于OOD检测。该方法不依赖对CLIP主干的微调，仅需一个小型K-shot缓存和轻量级超参数选择，内存开销随类别数C和样本数K呈O(CK²)增长。在ImageNet及多个OOD基准上的实验表明，GP-Adapter在少样本任务中表现优异，并与提示学习基线结合后持续提升OOD检测性能，凸显了基于概率推理与提示学习的互补性。结果表明，将概率推断融入大规模预训练视觉语言模型，可提升低数据与分布偏移场景下的可靠性。代码已开源。
原文摘要 · Abstract (English)
We propose GP-Adapter, a training-free framework that augments CLIP (Contrastive Language-Image Pre-training) with Gaussian Process (GP) uncertainty modeling for few-shot classification and out-of-distribution (OOD) detection. While CLIP achieves strong zero-shot recognition, it yields deterministic similarity scores and offers limited uncertainty information, which is critical under distribution shift and data scarcity. GP-Adapter constructs modality-specific, class-wise one-class GPs on top of frozen CLIP embeddings using an RBF kernel for image features and a linear kernel for text prompts and fuses their predictive statistics to produce a variance-aware confidence score for OOD detection. The method requires no fine-tuning of the CLIP backbone and relies only on a small $K$-shot cache and lightweight hyperparameter selection, with memory cost scaling as $O(CK^2)$ for $C$ classes and $K$ shots. Experiments on ImageNet and multiple OOD benchmarks show that GP-Adapter provides competitive few-shot performance and consistently improves OOD detection when combined with prompt-learning baselines, highlighting the complementarity between GP-based uncertainty modeling and prompt learning. Overall, our results suggest that integrating probabilistic inference with large pre-trained vision-language models can improve reliability in low-data and distribution-shifted settings. Code is available at https://github.com/tms-byte/GP-Adapter
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。