arXiv:2506.02367cs.CV2025-06

用神经场替换ViT分类头,显著降低训练成本并提升新类别识别能力。

ViTNF: Leveraging Neural Fields to Boost Vision Transformers in Generalized Category Discovery

  • 用两个耦合的静态神经场替代MLP分类头,分别存储特征与类别信息。
  • 在CIFAR-100等数据集上新类准确率提升19%,全类准确率提升16%。
  • 适合需要少样本、低训练成本的新类别发现场景,尤其适用于开放世界识别。

通用类别发现(GCD)是开放世界识别中的热门任务,旨在利用已知类别数据识别未知类别样本。现有Vision Transformer(ViT)通过预训练、元训练和微调实现优秀少样本学习能力,但其MLP分类头与网络同步训练,增加训练难度与成本,未能充分释放特征提取器潜力。本文提出将MLP头替换为基于神经场的分类器(NF),设计一种新的静态神经场函数描述神经场激活分布,并构建由两个静态神经场组成的高效少样本分类器。该神经场分类器通过基础场存储支持样本特征,高层场表示已知类别,交叉场连接记录支持样本类别信息。将原模型中的MLP头替换为该神经场分类器,形成新架构ViTNF。通过先在源任务上预训练特征提取器,再在元测试阶段单独训练神经场分类器,简化三阶段训练流程,显著降低对训练样本的需求与训练难度。为进一步提升新类别识别能力,提出有效算法确定基础场横向交互尺度。实验表明,模型在CIFAR-100、ImageNet-100、CUB-200和Standard Cars数据集上超越现有最先进方法,在新类别和全部类别上分别取得19%和16%的准确率提升,展现出在通用类别发现任务中的显著优势。

原文摘要 · Abstract (English)

Generalized category discovery (GCD) is a highly popular task in open-world recognition, aiming to identify unknown class samples using known class data. By leveraging pre-training, meta-training, and fine-tuning, ViT achieves excellent few-shot learning capabilities. Its MLP head is a feedforward network, trained synchronously with the entire network in the same process, increasing the training cost and difficulty without fully leveraging the power of the feature extractor. This paper proposes a new architecture by replacing the MLP head with a neural field-based one. We first present a new static neural field function to describe the activity distribution of the neural field and then use two static neural field functions to build an efficient few-shot classifier. This neural field-based (NF) classifier consists of two coupled static neural fields. It stores the feature information of support samples by its elementary field, the known categories by its high-level field, and the category information of support samples by its cross-field connections. We replace the MLP head with the proposed NF classifier, resulting in a novel architecture ViTNF, and simplify the three-stage training mode by pre-training the feature extractor on source tasks and training the NF classifier with support samples in meta-testing separately, significantly reducing ViT's demand for training samples and the difficulty of model training. To enhance the model's capability in identifying new categories, we provide an effective algorithm to determine the lateral interaction scale of the elementary field. Experimental results demonstrate that our model surpasses existing state-of-the-art methods on CIFAR-100, ImageNet-100, CUB-200, and Standard Cars, achieving dramatic accuracy improvements of 19\% and 16\% in new and all classes, respectively, indicating a notable advantage in GCD.

视觉变压器神经场少样本学习开放世界识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。