利用图像空间结构提升文本生成图像的推理速度
Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation
- 通过建模图像二维空间结构，优化草稿模型预测
- 在多个基准上实现1.71倍加速，保持图像质量和多样性
- 适合需要快速生成高质量图像的研究与应用
自回归图像生成模型虽能生成高保真图像，但因逐标记解码过程导致推理缓慢。推测解码技术通过轻量级草稿模型近似大模型输出，在文本生成中已展现加速潜力，但在图像生成领域仍不充分。挑战源于采样空间庞大，难以对齐草稿与目标模型输出，且未有效利用图像固有的二维空间结构，限制了局部依赖建模。为此，我们提出Hawk，利用图像空间结构引导草稿模型进行更准确高效的预测。在多个文本到图像基准上的实验表明，相比标准自回归模型，Hawk实现1.71倍加速，同时保持图像保真度和多样性。
原文摘要 · Abstract (English)
Autoregressive (AR) image generation models are capable of producing high-fidelity images but often suffer from slow inference due to their inherently sequential, token-by-token decoding process. Speculative decoding, which employs a lightweight draft model to approximate the output of a larger AR model, has shown promise in accelerating text generation without compromising quality. However, its application to image generation remains largely underexplored. The challenges stem from a significantly larger sampling space, which complicates the alignment between the draft and target model outputs, coupled with the inadequate use of the two-dimensional spatial structure inherent in images, thereby limiting the modeling of local dependencies. To overcome these challenges, we introduce Hawk, a new approach that harnesses the spatial structure of images to guide the speculative model toward more accurate and efficient predictions. Experimental results on multiple text-to-image benchmarks demonstrate a 1.71x speedup over standard AR models, while preserving both image fidelity and diversity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。