利用图像空间相关性并行生成，加速自回归图像模型推理
SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation
- 在解码时同时预测相邻水平和下方像素，实现多方向并行
- 推理速度最高提升11.03倍，保持生成质量不变
- 适合追求高效图像生成的开发者与研究者
自回归图像模型将图像视为一维标记序列，继承了语言模型的逐标记生成方式。这种展开方式忽略了图像的重要特性：相邻标记在二维空间上具有相关性。我们提出空间推测解码（SSD），一种推理时的解码框架，利用图像的这种空间结构。与仅沿展开序列推测不同，SSD同时预测相邻的水平标记和正下方的标记，使多个空间方向可并行推进。这减少了主干网络前向传播次数，并缓解了自回归解码的内存瓶颈。在DPG-Bench和GenEval数据集上，SSD将图像生成速度最高提升11.03倍，同时保持生成质量。结果表明，空间结构为自回归图像生成提供了一种简单而有效的并行机制。
原文摘要 · Abstract (English)
Autoregressive image models treat images as 1D token sequences, inheriting the next-token factorization of language models. This flattening discards a useful property of images: nearby tokens are correlated in two dimensions, not one. We introduce Spatially Speculative Decoding (SSD), an inference-time decoding framework that exploits this spatial structure. Rather than speculating only along the flattened sequence, SSD predicts both the adjacent horizontal token and the token directly below it, allowing multiple spatial directions to advance in parallel. This reduces the number of backbone forward evaluations and alleviates the memory bottleneck of autoregressive decoding. SSD accelerates image generation by up to 11.03x in wall-clock time while maintaining generation quality on DPG-Bench and GenEval. These results show that spatial structure provides a simple and effective source of parallelism for autoregressive image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。