SJD++让文本生成图像更快，一步预测多个词元，不需训练。
SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image Generation
- 每步预测多个词元，结合雅可比迭代与概率抽样机制。
- 推理延迟降低2到3倍，生成步数压缩2到7倍。
- 适合追求速度的图像生成应用，无需额外训练。
大规模自回归模型虽能生成高质量高分辨率图像，但推理速度缓慢，需数百至数千次串行前向计算进行下一步词元预测。为加速自回归文本到图像生成，我们提出训练无关的改进型推测雅可比解码（SJD++），一种概率并行解码算法。不同于传统逐词元预测，SJD++在每次前向传播中执行多词元预测，显著减少生成步骤。具体而言，它融合了雅可比解码的迭代多词元预测机制与推测采样的概率起草-验证机制。更重要的是，为实现进一步加速，SJD++在每次验证后复用高置信度草稿词元，而非重新采样全部。我们在多个代表性自回归文本到图像生成模型上进行了广泛实验，结果表明SJD++实现了2到3倍的推理延迟降低和2到7倍的步数压缩，同时保持视觉质量无明显下降。
原文摘要 · Abstract (English)
Large autoregressive models can generate high-quality, high-resolution images but suffer from slow generation speed, because these models require hundreds to thousands of sequential forward passes for next-token prediction during inference. To accelerate autoregressive text-to-image generation, we propose Speculative Jacobi Decoding++ (SJD++), a training-free probabilistic parallel decoding algorithm. Unlike traditional next-token prediction, SJD++ performs multi-token prediction in each forward pass, drastically reducing generation steps. Specifically, it integrates the iterative multi-token prediction mechanism from Jacobi decoding, with the probabilistic drafting-and-verification mechanism from speculative sampling. More importantly, for further acceleration, SJD++ reuses high-confidence draft tokens after each verification phase instead of resampling them all. We conduct extensive experiments on several representative autoregressive text-to-image generation models and demonstrate that SJD++ achieves $2\times$ to $3\times$ inference latency reduction and $2\times$ to $7\times$ step compression, while preserving visual quality with no observable degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。