移除位置编码和分类令牌，让小模型更适配医学图像的弱空间结构。
ZACH-ViT: Regime-Dependent Inductive Bias in Compact Vision Transformers for Medical Imaging
- 不用位置编码和[CLS] token，用全局平均池化实现对图像块的无序处理。
- 在7个医学数据集上仅用50样本/类时，参数量0.25M仍表现优异。
- 适合数据少、计算资源有限的医学图像分析场景。
视觉变换器依赖位置编码和分类标记来编码固定的先验空间信息。尽管在自然图像中有效，但这些先验在空间布局信息较弱的医学影像中可能不适用。本文提出ZACH-ViT（零标记自适应紧凑分层视觉变换器），一种移除位置编码和[CLS]标记的紧凑视觉变换器，通过全局平均池化实现对图像块的排列不变性处理。零标记指去除专用聚合标记和位置编码，图像块标记保持不变。自适应残差投影在严格参数约束下保持训练稳定性。我们在七种MedMNIST数据集上采用严格的少样本协议（每类50样本，固定超参数，五次随机种子）评估。结果表明其行为具有依赖于数据分布的特性：在BloodMNIST上优势最显著，在PathMNIST上仍具竞争力，而在OCTMNIST和OrganAMNIST等具有更强解剖先验的数据集上相对优势下降，与假设一致。组件与池化消融实验显示，随着空间结构增强，位置支持逐渐有益；而重新引入[CLS]标记始终不利。这些发现表明，架构与数据结构的匹配可超越通用基准性能。尽管参数极少且无预训练，ZACH-ViT在数据稀缺条件下仍表现良好，适用于紧凑医学影像与低资源场景。代码见：https://github.com/Bluesman79/ZACH-ViT
原文摘要 · Abstract (English)
Vision Transformers rely on positional embeddings and class tokens encoding fixed spatial priors. While effective for natural images, these priors may be suboptimal when spatial layout is weakly informative, a frequent condition in medical imaging. We introduce ZACH-ViT (Zero-token Adaptive Compact Hierarchical Vision Transformer), a compact Vision Transformer that removes positional embeddings and the [CLS] token, achieving permutation-invariant patch processing via global average pooling. Zero-token denotes removal of the dedicated aggregation token and positional encodings. Patch tokens remain unchanged. Adaptive residual projections preserve training stability under strict parameter constraints. We evaluate ZACH-ViT across seven MedMNIST datasets under a strict few-shot protocol (50 samples/class, fixed hyperparameters, five seeds). Results reveal regime-dependent behavior: ZACH-ViT (0.25M parameters, trained from scratch) achieves strongest advantage on BloodMNIST and remains competitive on PathMNIST, while relative advantage decreases on datasets with stronger anatomical priors (OCTMNIST, OrganAMNIST), consistent with our hypothesis. Component and pooling ablations show positional support becomes mildly beneficial as spatial structure increases, whereas reintroducing a [CLS] token is consistently unfavorable. These findings support that architectural alignment with data structure can outweigh universal benchmark dominance. Despite minimal size and no pretraining, ZACH-ViT achieves competitive performance under data-scarce conditions, relevant for compact medical imaging and low-resource settings. Code: https://github.com/Bluesman79/ZACH-ViT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。