arXiv:2503.10568cs.CV2025-03被引 18

提出随机并行解码机制,让图像自回归生成更快更省内存。

Autoregressive Image Generation with Randomized Parallel Decoding

  • 分离位置引导与内容表征,实现完全随机顺序生成
  • 32步采样即达FID 1.83,推理速度提升30倍以上
  • 支持零样本修复与分辨率扩展,适合高效生成任务

我们提出ARPG,一种新型视觉自回归模型,支持随机并行生成,克服了传统栅格顺序方法在推理效率和零样本泛化上的局限。核心思路是:有效随机顺序建模需显式指导下一个预测位置。为此,我们设计了一种解耦解码框架,将位置引导与内容表示分别编码为查询和键值对,并直接融入因果注意力机制,实现全随机顺序训练与生成,无需双向注意力。由此,ARPG可轻松泛化至零样本任务,如图像修复、外推和分辨率提升。此外,通过共享键值缓存并行处理多个查询,支持高效并行推理。在ImageNet-1K 256基准上,仅用32次采样即达到FID 1.83,相比同类模型推理速度提升超30倍,内存消耗降低75%。

原文摘要 · Abstract (English)

We introduce ARPG, a novel visual Autoregressive model that enables Randomized Parallel Generation, addressing the inherent limitations of conventional raster-order approaches, which hinder inference efficiency and zero-shot generalization due to their sequential, predefined token generation order. Our key insight is that effective random-order modeling necessitates explicit guidance for determining the position of the next predicted token. To this end, we propose a novel decoupled decoding framework that decouples positional guidance from content representation, encoding them separately as queries and key-value pairs. By directly incorporating this guidance into the causal attention mechanism, our approach enables fully random-order training and generation, eliminating the need for bidirectional attention. Consequently, ARPG readily generalizes to zero-shot tasks such as image in-painting, out-painting, and resolution expansion. Furthermore, it supports parallel inference by concurrently processing multiple queries using a shared KV cache. On the ImageNet-1K 256 benchmark, our approach attains an FID of 1.83 with only 32 sampling steps, achieving over a 30 times speedup in inference and and a 75 percent reduction in memory consumption compared to representative recent autoregressive models at a similar scale.

自回归生成并行推理图像生成高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。