用ALiBi编码降低ViT位置偏见，提升显微图像分割效果
What DINO saw: ALiBi positional encoding reduces positional bias in Vision Transformers
- 改用ALiBi相对位置编码，缓解ViT的位置偏好问题
- 在多种任务和编码方式下验证了位置偏差的存在性
- 适用于材料科学等无方向偏好图像的零样本迁移
视觉变换器（ViTs）尤其是像DINOv2这样的特征基础模型，能学习到对多种下游任务有用的丰富表征。然而，架构选择（如位置编码）可能导致模型表现出与语义内容无关的位置偏差和伪影，这使得在材料科学等领域的零样本适配变得困难，因为这类图像通常是均质微观结构的横截面（即无特定方向）。本文通过线性探测研究了ViTs中的位置偏差，发现其存在于多种目标和位置编码中，并通过微调使模型采用ALiBi相对位置编码来减少该偏差。我们证明这些模型保持了理想的通用语义表征，且其无偏特征可成功用于复杂显微图像的可训练分割。
原文摘要 · Abstract (English)
Vision transformers (ViTs) - especially feature foundation models like DINOv2 - learn rich representations useful for many downstream tasks. However, architectural choices (such as positional encoding) can lead to these models displaying positional biases and artefacts independent of semantic content. This makes zero-shot adaption difficult in fields like material science, where images are often cross-sections of homogeneous microstructure (i.e. having no preferred direction). In this work, we investigate the positional bias in ViTs via linear probing, finding it present across a range of objectives and positional encodings, and subsequently reduce it by finetuning models to use ALiBi relative positional encoding. We demonstrate that these models retain desirable general semantics and their unbiased features can be used successfully in trainable segmentation of complex microscopy images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。