Iwin Transformer通过交错窗口与卷积结合，解决视觉Transformer的复杂度和跨分辨率训练难题。
Iwin Transformer: Hierarchical Vision Transformer using Interleaved Windows
- 用交错窗口注意力加深度卷积替代位置编码，实现无位置嵌入的层级结构。
- 224→384分辨率微调使Iwin-L在ImageNet-1K上准确率从86.4%提升至87.4%。
- 支持低到高分辨率微调及2D到3D权重迁移，适合多任务与跨模态应用。
视觉变压器（ViTs）存在两个局限：位置编码的固定分辨率依赖性，阻碍了跨分辨率微调；以及注意力机制的二次复杂度。尽管Swin Transformer通过窗口注意力缓解了后者，但其微调能力受限。遵循“无孤立标记”的理念，我们提出Iwin Transformer，一种无位置编码的层级视觉变压器，将交错窗口注意力与深度卷积集成于单一模块中。注意力捕捉长程依赖，卷积连接局部邻居并隐式编码空间位置。该设计不仅降低注意力的二次复杂度，还实现两种可扩展性：从低到高分辨率的微调，以及从2D到3D的权重迁移。仅通过调整窗口大小，直接将Iwin-L从224²微调至384²，在ImageNet-1K上准确率由86.4%提升至87.4%。将ImageNet预训练的Iwin-T迁移至视频任务，在Kinetics-400上达到79.1%，优于Swin-T（78.8%），且减少15.9%的FLOPs。Iwin在ADE20K分割与条件图像生成（FlashDiT）任务上也保持竞争力。总体而言，Iwin为同时应对ViTs的复杂度与可扩展性挑战提供有效方案。代码与模型见https://github.com/cominder/Iwin-Transformer。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) face two limitations: the rigid resolution dependency of positional embeddings, which complicates cross-resolution fine-tuning, and the quadratic complexity of attention. While Swin Transformer alleviates the latter through window attention, it suffers from fine-tuning. Following the philosophy "no token is an island," we present Iwin Transformer, a position-embedding-free hierarchical vision transformer that couples interleaved window attention with depthwise convolution inside a single block. Attention captures long-range dependencies, while convolution links local neighbors and implicitly encodes spatial position. This design not only reduces the quadratic complexity of attention but also enables two types of scalability: fine-tuning from low to high resolution and weight transfer from 2D to 3D. With window-size adjustment alone, direct $224^2{\rightarrow}384^2$ fine-tuning lifts Iwin-L from 86.4\% to 87.4\% top-1 accuracy on ImageNet-1K. Transferring an ImageNet-pretrained Iwin-T to video achieves 79.1\% on Kinetics-400, outperforming Swin-T (78.8\%) with 15.9\% fewer FLOPs. Iwin also remains competitive on ADE20K segmentation and class-conditional image generation (FlashDiT). Overall, Iwin offers an effective approach to simultaneously tackling the complexity and scalability challenges in ViTs. Code and models are at https://github.com/cominder/Iwin-Transformer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。