arXiv:2510.08994cs.CV2025-10NeurIPS被引 8

通过噪声预测加速文本到图像生成,减少上千次推理步骤。

Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generation

  • 用噪声嵌入预测下一干净标记,实现并行生成。
  • 仅需少量前向传播(<1000次)生成高质量图像。
  • 适合需要快速生成的图像应用开发者。

自回归文本到图像模型因逐标记生成导致推理缓慢,通常需数千次前向传播生成一张图。为此,我们提出推测性雅可比去噪解码(SJD2),将去噪过程融入雅可比迭代,使自回归模型能并行生成标记。方法引入下一干净标记预测范式,使预训练模型可接受含噪声的标记嵌入,并通过低成本微调预测下一干净标记,引导模型走向更稳定的雅可比轨迹。推理时,以高斯噪声初始化标记序列,在嵌入空间中进行迭代的下一干净标记预测。采用概率准则并行验证与接受多个标记,并对未被接受的标记在去噪轨迹中优化。实验表明,该方法可显著减少前向传播次数,同时保持生成图像的视觉质量。

原文摘要 · Abstract (English)

As a new paradigm of visual content generation, autoregressive text-to-image models suffer from slow inference due to their sequential token-by-token decoding process, often requiring thousands of model forward passes to generate a single image. To address this inefficiency, we propose Speculative Jacobi-Denoising Decoding (SJD2), a framework that incorporates the denoising process into Jacobi iterations to enable parallel token generation in autoregressive models. Our method introduces a next-clean-token prediction paradigm that enables the pre-trained autoregressive models to accept noise-perturbed token embeddings and predict the next clean tokens through low-cost fine-tuning. This denoising paradigm guides the model towards more stable Jacobi trajectories. During inference, our method initializes token sequences with Gaussian noise and performs iterative next-clean-token-prediction in the embedding space. We employ a probabilistic criterion to verify and accept multiple tokens in parallel, and refine the unaccepted tokens for the next iteration with the denoising trajectory. Experiments show that our method can accelerate generation by reducing model forward passes while maintaining the visual quality of generated images.

图像生成加速推理去噪解码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。