Atlas用多尺度注意力提升长图像建模效率，速度快且精度高。
Atlas: Multi-Scale Attention Improves Long Context Image Modeling
- 构建多尺度特征并双向传递信息，实现跨尺度通信
- 在1024px下准确率达91.04%，比ConvNext-B快4.3倍
- 适合高分辨率图像任务，尤其擅长长上下文建模
高效建模大规模图像是机器学习中的长期挑战。为此，我们提出多尺度注意力（MSA），其核心思想为多尺度表示与双向跨尺度通信。MSA生成O(log N)个尺度的图像特征，并通过交叉注意力实现跨尺度信息传递。基于MSA，我们设计了新架构Atlas。在高分辨率ImageNet-100上，Atlas显著优化了计算性能权衡：在1024px分辨率下，Atlas-B准确率达91.04%，接近ConvNext-B的91.92%，但速度提升4.3倍；相比FasterViT快2.95倍、准确率高7.38%，比LongViT快2.25倍、准确率高4.96%。与MambaVision-S对比，在1024px、2048px、4096px分辨率下，Atlas-S分别高出5%、16%、32%准确率，运行时间相当。代码与预训练模型已开源。
原文摘要 · Abstract (English)
Efficiently modeling massive images is a long-standing challenge in machine learning. To this end, we introduce Multi-Scale Attention (MSA). MSA relies on two key ideas, (i) multi-scale representations (ii) bi-directional cross-scale communication. MSA creates O(log N) scales to represent the image across progressively coarser features and leverages cross-attention to propagate information across scales. We then introduce Atlas, a novel neural network architecture based on MSA. We demonstrate that Atlas significantly improves the compute-performance tradeoff of long-context image modeling in a high-resolution variant of ImageNet 100. At 1024px resolution, Atlas-B achieves 91.04% accuracy, comparable to ConvNext-B (91.92%) while being 4.3x faster. Atlas is 2.95x faster and 7.38% better than FasterViT, 2.25x faster and 4.96% better than LongViT. In comparisons against MambaVision-S, we find Atlas-S achieves 5%, 16% and 32% higher accuracy at 1024px, 2048px and 4096px respectively, while obtaining similar runtimes. Code for reproducing our experiments and pretrained models is available at https://github.com/yalalab/atlas.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。