轻量级模块让ViT更聚焦关键部位，提升细粒度图像分类准确率
The Loupe: A Plug-and-Play Attention Module for Amplifying Discriminative Features in Vision Transformers
- 在ViT中间层插入小CNN生成空间掩码，动态重加权特征
- CUB数据集上提升Swin-Base精度至91.72%，参数增益不足0.1%
- 适合需要精准定位的细粒度视觉任务，不依赖部件标注
细粒度视觉分类要求模型关注细微、任务相关的区域而非整体对象上下文。我们提出The Loupe，一种轻量级即插即用的空间门控模块，用于分层Vision Transformers。该模块插入中间特征阶段，通过小型CNN预测单通道空间掩码，并在端到端训练中使用交叉熵目标与L1稀疏正则项，对特征激活进行重加权。在CUB-200-2011数据集上，The Loupe将Swin-Base准确率从88.36%提升至91.72%，Swin-Tiny从85.14%提升至88.61%，额外参数少于0.1%。消融实验表明，性能提升依赖插入位置和稀疏正则项，说明受控空间门控比简单多尺度掩码更有效。定性结果显示，学习到的掩码常与鸟类关键部位对齐，但该模块无法替代部件级监督，在遮挡或细粒度内部差异下可能失效。
原文摘要 · Abstract (English)
Fine-Grained Visual Classification (FGVC) requires models to focus on subtle, task-relevant regions rather than broad object context. We present The Loupe, a lightweight plug-and-play spatial gating module for hierarchical Vision Transformers. The module is inserted at an intermediate feature stage, predicts a single-channel spatial mask with a small CNN, and uses that mask to reweight feature activations during end-to-end training with a cross-entropy objective and an l1 sparsity term. On CUB-200-2011, The Loupe improves Swin-Base from 88.36% to 91.72% and Swin-Tiny from 85.14% to 88.61%, with under 0.1% additional parameters. Ablations show that the improvement depends on the insertion point and the sparsity regularizer, suggesting that controlled spatial gating is more effective than naive multi-scale masking in this setting. Qualitative results indicate that the learned masks often align with discriminative bird parts, although the module is not a substitute for part-level supervision and can fail under occlusion or fine-grained intra-part differences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。