让视觉Transformer无视图像大小，高效预训练。
Self-supervised pretraining for an iterative image size agnostic vision transformer
- 用迭代多尺度块处理图像，实现分辨率无关的视觉编码。
- 在ImageNet-1K上达到与现有方法相当的分类性能。
- 计算开销恒定，适合大图或变尺寸输入场景。
视觉变压器（ViT）在自监督学习（SSL）中占据主导地位。尽管其在大规模预训练中表现优异，但对图像尺寸的计算效率低下且扩展性差。因此，像DINO这样的基础模型受限于低分辨率处理。近期一种受视网膜启发的变压器通过迭代处理固定大小的多尺度补丁实现了分辨率无关性，该模型在监督学习中表现良好，采用无时间反向传播的序列化、类循环过程。为释放其作为基础主干的潜力，我们提出基于DINO自蒸馏目标的新型序列到全局自监督学习框架。依托高效的积分图像补丁提取方法，我们的方法支持图像尺寸无关视觉编码器的大规模预训练。我们在ImageNet-1K及下游分类任务中取得具有竞争力的表现，且计算预算不随输入分辨率变化。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) dominate self-supervised learning (SSL). While they have proven highly effective for large-scale pretraining, they are computationally inefficient and scale poorly with image size. Consequently, foundational models like DINO are constrained to low-resolution processing. A recent foveal-inspired transformer achieves resolution agnosticism by iteratively processing a fixed-size context of multi-zoom patches. This model demonstrated promising results via supervised learning, utilizing a sequential, recurrent-like process without backpropagation through time. To unlock its potential as a foundational backbone, we introduce a novel sequential-to-global SSL framework based on DINO's self-distillation objective. Supported by an efficient integral-image patch extraction method, our approach enables large-scale pretraining for image-size agnostic vision encoders. We achieve competitive performance on ImageNet-1K and downstream classification tasks, maintaining a constant computational budget regardless of input resolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。