用ViT小模型构建图像分类原型，5次学习下表现更优
ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation
- 将ViT-Small融入原型网络，用支持样本均值生成类别原型
- 在4个数据集上5次学习准确率提升最高达3.2%，特征分离更好
- 轻量级结构比同类Transformer方法更高效，适合快速复现
视觉变换器（ViT）强大的表征能力在少样本图像分类中仍未被充分挖掘。本文提出ViT-ProtoNet，将ViT-Small主干网络集成到原型网络框架中。通过平均少量支持样本的类别条件令牌嵌入，构建出在5次学习设置下可泛化至新类别的鲁棒原型。我们在四个标准基准（Mini-ImageNet、FC100、CUB-200、CIFAR-FS）上进行了广泛评估，包含重叠支持集变体以检验鲁棒性。在所有划分中，ViT-ProtoNet持续优于基于CNN的原型方法，5次学习准确率最高提升3.2%，且在潜在空间中表现出更优的特征可分性。此外，其性能优于或媲美使用更重型骨干网络的Transformer基线方法。全面消融实验分析了变换器深度、图像块大小及微调策略的影响。为促进可复现性，我们公开代码与预训练权重。结果表明，ViT-ProtoNet是少样本分类的一种强大且灵活的方法，并为基于Transformer的元学习者设立了新基准。
原文摘要 · Abstract (English)
The remarkable representational power of Vision Transformers (ViTs) remains underutilized in few-shot image classification. In this work, we introduce ViT-ProtoNet, which integrates a ViT-Small backbone into the Prototypical Network framework. By averaging class conditional token embeddings from a handful of support examples, ViT-ProtoNet constructs robust prototypes that generalize to novel categories under 5-shot settings. We conduct an extensive empirical evaluation on four standard benchmarks: Mini-ImageNet, FC100, CUB-200, and CIFAR-FS, including overlapped support variants to assess robustness. Across all splits, ViT-ProtoNet consistently outperforms CNN-based prototypical counterparts, achieving up to a 3.2\% improvement in 5-shot accuracy and demonstrating superior feature separability in latent space. Furthermore, it outperforms or is competitive with transformer-based competitors using a more lightweight backbone. Comprehensive ablations examine the impact of transformer depth, patch size, and fine-tuning strategy. To foster reproducibility, we release code and pretrained weights. Our results establish ViT-ProtoNet as a powerful, flexible approach for few-shot classification and set a new baseline for transformer-based meta-learners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。