不靠注入位置编码,用训练时监督让ViT自动学会空间结构。
Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformers

- 训练时引入坐标回归损失,引导模型从数据中学习空间规律。
- 在ImageNet-100等任务上超越带位置编码的基线模型。
- 适合追求轻量架构、无需位置编码的视觉模型研究者。
视觉变换器(ViTs)通常依赖注入的位置机制来解决自注意力的排列不变性问题。受自然图像空间规律启发,我们探讨是否可通过数据驱动而非显式注入实现空间组织。在严格匹配的从头训练条件下,提出仅训练阶段使用的主动空间引导(Guidance)方法:禁用位置注入,对最后一层图块令牌施加辅助的二维坐标回归损失。引导头仅用于训练,推理时移除;部署模型为无位置注入的ViT编码器与任务特定预测模块。使用DINOv3 ViT主干网络,Guidance在ImageNet-100分类、ADE20K语义分割和Hypersim单目深度估计任务中持续提升性能,优于学习型绝对位置嵌入和旋转位置编码等强基线,且在相同训练协议下表现更优。在ImageNet-100上对代表性注入位置设计的扩展对比进一步验证其有效性。Guidance还增强了分辨率迁移下的鲁棒性,多分辨率训练进一步提升了跨输入尺寸的准确性。结果表明,ViT的空间归纳偏置无需架构注入,可通过训练时监督塑造。训练与评估代码已公开于https://github.com/cloudlc/asg。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) commonly rely on injected positional mechanisms to address self-attention's permutation invariance. Motivated by the spatial regularities of natural images, we ask whether spatial organization can be induced from data rather than explicitly injected. Under controlled, matched from-scratch training, we propose Active Spatial Guidance (Guidance), a training-only objective that disables positional injection and applies an auxiliary 2D coordinate-regression loss to the final-layer patch tokens. The guidance head is used only during training and removed for inference; the deployed model consists of a positional-injection-free ViT encoder and the task-specific prediction module. Using DINOv3 ViT backbones, Guidance consistently improves performance on ImageNet-100 classification, ADE20K semantic segmentation, and Hypersim monocular depth estimation, outperforming strong injected baselines such as learned absolute positional embeddings and rotary positional embeddings under identical training protocols. On ImageNet-100, broader comparisons against representative injected positional designs further support Guidance's effectiveness. Guidance also improves robustness under resolution transfer, and multi-resolution training further strengthens accuracy across input sizes. Overall, our results suggest that spatial inductive bias in ViTs need not be architecturally injected, but can be shaped through training-time supervision. The code used for training and evaluation is publicly available in https://github.com/cloudlc/asg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。