提出自适应位置嵌入，让ViT自动学习是否使用位置信息
Learning to Adapt to Position Bias in Vision Transformer Classifiers
- 用扩展的SHAP方法量化数据集的位置偏差程度
- 不同数据集位置偏差差异大，影响模型性能表现
- 新嵌入可动态调节位置信息强度，适合多种任务
图像分类中位置信息的重要性取决于数据分布。一方面相机位置随机，物体可出现在任意位置，需具备平移不变性；另一方面，捕捉偏置和场景布局（如天空在上方）依赖位置信息。本文发现，位置偏差——即利用输入特征的位置信息能多大程度提升分类效果——对Vision Transformer分类器性能有关键影响。为此提出Position-SHAP，通过扩展SHAP方法直接度量位置偏差。实验显示不同数据集存在显著位置偏差差异，并表明最优位置嵌入选择应匹配数据集的偏差水平。因此提出Auto-PE，一种单参数位置嵌入扩展，允许位置嵌入的范数自适应调节，从而实现位置信息的“遗忘”。Auto-PE可与现有位置嵌入结合，在多个分类数据集上达到或超越原有精度。
原文摘要 · Abstract (English)
How discriminative position information is for image classification depends on the data. On the one hand, the camera position is arbitrary and objects can appear anywhere in the image, arguing for translation invariance. At the same time, position information is key for exploiting capture/center bias, and scene layout, e.g.: the sky is up. We show that position bias, the level to which a dataset is more easily solved when positional information on input features is used, plays a crucial role in the performance of Vision Transformers image classifiers. To investigate, we propose Position-SHAP, a direct measure of position bias by extending SHAP to work with position embeddings. We show various levels of position bias in different datasets, and find that the optimal choice of position embedding depends on the position bias apparent in the dataset. We therefore propose Auto-PE, a single-parameter position embedding extension, which allows the position embedding to modulate its norm, enabling the unlearning of position information. Auto-PE combines with existing PEs to match or improve accuracy on classification datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。