零值替换会夸大视觉Transformer中寄存器对内容的依赖，实际只需类似激活即可。
Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers
- 用均值、噪声或跨图置换替代寄存器，性能几乎不变
- 零值替换导致分类性能下降36.6个百分点，远超其他方法
- 适合研究模型可解释性与特征敏感性的研究人员
零值替换（zero-ablation）常被用于探究视觉Transformer中令牌的功能。在DINOv2+registers和DINOv3中，该方法导致分类性能下降最高达-36.6个百分点，分割性能下降-30.9个百分点，暗示寄存器功能不可或缺。然而，三种替代控制实验——均值替换、噪声替换和跨图像寄存器洗牌——在分类、对应关系和分割任务中均保持性能稳定，仅比原始基线低约1个百分点。逐标记余弦相似度分析显示，这些替代确实扰动了内部表示，而零值替换造成的扰动异常剧烈，解释了为何其性能下降显著。结论：零值替换过度夸大了对寄存器精确内容的依赖。在冻结特征评估中，性能取决于合理的寄存器类激活，而非具体图像值。寄存器仍能缓解[CLS]对密集特征的依赖，并关联压缩的补丁几何结构。这些发现，包括替代控制结果，在ViT-B规模下可复现。
原文摘要 · Abstract (English)
Zero-ablation -- replacing token activations with zero vectors -- is widely used to probe token function in vision transformers. Register zeroing in DINOv2+registers and DINOv3 produces large drops (up to $-36.6$\,pp classification, $-30.9$\,pp segmentation), suggesting registers are functionally indispensable. However, three replacement controls -- mean-substitution, noise-substitution, and cross-image register-shuffling -- preserve performance across classification, correspondence, and segmentation, remaining within ${\sim}1$\,pp of the unmodified baseline. Per-patch cosine similarity shows these replacements genuinely perturb internal representations, while zeroing causes disproportionately large perturbations, consistent with why it alone degrades tasks. We conclude that zero-ablation overstates dependence on exact register content. In the frozen-feature evaluations we test, performance depends on plausible register-like activations rather than on exact image-specific values. Registers nevertheless buffer dense features from \texttt{[CLS]} dependence and are associated with compressed patch geometry. These findings, including the replacement-control results, replicate at ViT-B scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。