arXiv:2607.09086cs.CV2026-07

让视觉Transformer更关注局部细节,提升细粒度识别精度。

Subtoken Vision Transformer for Fine-grained Recognition

论文配图:Subtoken Vision Transformer for Fine-grained Recognition
图 1 · 摘自论文原文
  • 用子令牌精细表示关键图像区域,保留全局上下文
  • 在CUB等数据集上将新类别准确率从81.3%提升至84.7%
  • 仅增加0.5毫秒延迟,适合实际部署

我们提出子令牌视觉Transformer(SubViT),一种面向细粒度视觉识别的可选图像标记方法。标准ViT将每个固定大小的图像块压缩为单一标记,但细粒度区分常依赖少数块内的局部变化。SubViT通过在判别性块中使用多个子令牌来弥补这一差距,同时保留原始标记序列以维持全局上下文,从而在最需要的地方分配额外容量。由于注意力头编码互补语义,且推理时提取注意力图需额外前向传播,我们采用两阶段训练策略:第一阶段使用随机注意力头采样的分割区域微调ViT,使模型接触多样化分割模式;第二阶段通过特征退化距离识别有信息量的注意力图,并将其蒸馏为轻量级单图路由器,直接预测确定性的标记重要性得分,无需独立注意力前向。我们在广义类别发现(GCD)任务上评估SubViT,该任务要求兼具细粒度区分能力和对未标记新类别的泛化能力。在CUB、FGVC-Aircraft和Stanford Cars上,SubViT将DINOv2的平均新类别准确率从81.3%提升至84.7%,仅增加0.50毫秒延迟和3.4%浮点运算量,同时相对于Retina Patch降低73.8%延迟。在CIFAR-10和ImageNet-100上的结果也证明了其广泛适用性。

原文摘要 · Abstract (English)

We present Subtoken Vision Transformer (SubViT), a selective image tokenization method for fine-grained visual recognition. Standard Vision Transformers compress each fixed-size patch into a single token, although fine-grained distinctions often depend on localized variations within only a few patches. SubViT addresses this mismatch by representing discriminative patches with multiple subtokens while retaining the original token sequence for global context, thereby allocating additional capacity where it is most needed. Since attention heads encode complementary semantics and extracting attention maps at inference requires an extra backbone forward, we adopt a two-stage training strategy. Stage 1 fine-tunes the ViT using subdivision regions sampled from random attention heads, exposing the model to diverse subdivision patterns. Stage 2 identifies informative attention maps through feature-degradation distances and distills them into a lightweight single-map router, which directly predicts deterministic token-importance scores without a separate attention forward. We evaluate SubViT on Generalized Category Discovery (GCD), a challenging task requiring both fine-grained discrimination and generalization to unlabeled novel categories. Across CUB, FGVC-Aircraft, and Stanford Cars, SubViT improves the average novel-category accuracy of DINOv2 from $81.3\%$ to $84.7\%$, with only $0.50$ ms additional latency and $3.4\%$ more FLOPs, while reducing latency by $73.8\%$ relative to Retina Patch. Results on CIFAR-10 and ImageNet-100 demonstrate its broader applicability.

细粒度识别视觉Transformer子令牌高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。