新注意力机制GSPN直接处理图像空间结构，提速超80倍。
Parallel Sequence Modeling via Generalized Spatial Propagation Network
- 用线扫描构建二维密集连接，保持空间一致性
- 序列长度压缩至√N，生成16K图像提速84倍
- 无需位置编码，适合图像分类与文生图任务
我们提出广义空间传播网络（GSPN），一种专为视觉任务优化的新注意力机制，能天然捕捉二维空间结构。现有注意力模型如Transformer、线性注意力和Mamba等将多维数据视为一维序列，损害空间连贯性与效率。GSPN通过线扫描方法直接在空间一致的图像数据上操作，建立密集的成对连接。其核心是稳定性-上下文条件，确保2D序列中稳定且上下文感知的传播，使正方形图像（含N个元素）的有效序列长度降至√N，显著提升计算效率。GSPN采用可学习的输入相关权重，无需位置嵌入，在ImageNet分类、类别引导图像生成及文生图任务中表现卓越。特别地，生成16K图像时，GSPN相比softmax注意力加速超过84倍。
原文摘要 · Abstract (English)
We present the Generalized Spatial Propagation Network (GSPN), a new attention mechanism optimized for vision tasks that inherently captures 2D spatial structures. Existing attention models, including transformers, linear attention, and state-space models like Mamba, process multi-dimensional data as 1D sequences, compromising spatial coherence and efficiency. GSPN overcomes these limitations by directly operating on spatially coherent image data and forming dense pairwise connections through a line-scan approach. Central to GSPN is the Stability-Context Condition, which ensures stable, context-aware propagation across 2D sequences and reduces the effective sequence length to $\sqrt{N}$ for a square map with N elements, significantly enhancing computational efficiency. With learnable, input-dependent weights and no reliance on positional embeddings, GSPN achieves superior spatial fidelity and state-of-the-art performance in vision tasks, including ImageNet classification, class-guided image generation, and text-to-image generation. Notably, GSPN accelerates SD-XL with softmax-attention by over $84\times$ when generating 16K images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。