用预训练先验提升ViT的分布外检测能力，无需额外数据或结构改动。
PViT: Prior-augmented Vision Transformer for Out-of-distribution Detection
- 基于预训练模型的类别置信度构建先验，引导ViT学习更鲁棒的决策边界。
- 在ImageNet与七种OOD数据集上，FPR95和AUROC均显著优于现有方法。
- 适用于希望增强ViT鲁棒性的研究者，尤其适合部署场景下的异常检测。
视觉变换器（ViTs）在多种视觉任务中取得显著成果，但其对数据分布偏移的鲁棒性及内在归纳偏置仍待深入探索。为提升ViT模型在图像分布外（OOD）检测中的表现，本文提出一种新颖且通用的框架——先验增强视觉变换器（PViT）。该方法以预训练模型输出的类别先验置信度为输入，训练PViT预测真实类别置信度。推理时，通过比较预测置信度与预训练先验置信度之间的差异来识别分布外样本。与现有SOTA方法不同，PViT利用所提出的先验引导置信度来塑造类别内与分布外之间的决策边界，无需额外的数据建模、生成方法或结构修改。在大规模ImageNet基准上，针对超过七种分布外数据集的大量实验表明，PViT在FPR95和AUROC指标上均显著优于现有最先进方法。代码已公开于https://github.com/RanchoGoose/PViT。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) have achieved remarkable success over various vision tasks, yet their robustness against data distribution shifts and inherent inductive biases remain underexplored. To enhance the robustness of ViT models for image Out-of-Distribution (OOD) detection, we introduce a novel and generic framework named Prior-augmented Vision Transformer (PViT). Taking as input the prior class logits from a pretrained model, we train PViT to predict the class logits. During inference, PViT identifies OOD samples by quantifying the divergence between the predicted class logits and the prior logits obtained from pre-trained models. Unlike existing state-of-the-art(SOTA) OOD detection methods, PViT shapes the decision boundary between ID and OOD by utilizing the proposed prior guided confidence, without requiring additional data modeling, generation methods, or structural modifications. Extensive experiments on the large-scale ImageNet benchmark, evaluated against over seven OOD datasets, demonstrate that PViT significantly outperforms existing SOTA OOD detection methods in terms of FPR95 and AUROC. The codebase is publicly available at https://github.com/RanchoGoose/PViT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。