通过感知局部性的并行解码，大幅加速自回归图像生成。
Locality-aware Parallel Decoding for Efficient Autoregressive Image Generation
- 设计可灵活并行的架构，支持任意生成顺序和并行度。
- 将256×256图像生成步数从256降至20，512×512降至48。
- 在不损失质量前提下，延迟降低至少3.4倍，适合高效生成场景。
我们提出局部感知并行解码（LPD）以加速自回归图像生成。传统方法依赖逐块预测，内存密集导致高延迟。现有工作尝试通过多块预测实现并行化，但并行度有限。为在保持生成质量的同时实现高并行性，我们引入两项关键技术：(1) 灵活并行自回归建模，一种新型架构，支持任意生成顺序与并行度；通过可学习的位置查询令牌引导目标位置生成，并确保并发生成块之间的相互可见性，实现一致并行解码。(2) 局部感知生成调度，将生成块分组以最小化组内依赖、最大化上下文支持，提升生成质量。在ImageNet类别条件生成任务上，256×256分辨率下生成步数从256降至20，512×512分辨率下从1024降至48，且未牺牲质量，相比先前并行化自回归模型延迟降低至少3.4倍。
原文摘要 · Abstract (English)
We present Locality-aware Parallel Decoding (LPD) to accelerate autoregressive image generation. Traditional autoregressive image generation relies on next-patch prediction, a memory-bound process that leads to high latency. Existing works have tried to parallelize next-patch prediction by shifting to multi-patch prediction to accelerate the process, but only achieved limited parallelization. To achieve high parallelization while maintaining generation quality, we introduce two key techniques: (1) Flexible Parallelized Autoregressive Modeling, a novel architecture that enables arbitrary generation ordering and degrees of parallelization. It uses learnable position query tokens to guide generation at target positions while ensuring mutual visibility among concurrently generated tokens for consistent parallel decoding. (2) Locality-aware Generation Ordering, a novel schedule that forms groups to minimize intra-group dependencies and maximize contextual support, enhancing generation quality. With these designs, we reduce the generation steps from 256 to 20 (256$\times$256 res.) and 1024 to 48 (512$\times$512 res.) without compromising quality on the ImageNet class-conditional generation, and achieving at least 3.4$\times$ lower latency than previous parallelized autoregressive models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。