通过感知图像内容动态调整生成速度与质量，提升自回归图像生成效率。
CSD: Content-aware Speculative Decoding for Efficient Image Generation
- 根据图像区域信息熵动态调节候选词接受概率，提升低细节区域生成速度。
- 在Lumina-mGPT和Janus-Pro上实现更高接受率，同时保持高质量输出。
- 适合需要高效图像生成的场景，如实时创作与大规模图像合成。
自回归图像生成中的推测解码（SD）虽能加速推理，但面临接受率低的问题，直接放宽标准又会降低图像质量。本文提出一种新的内容感知推测解码算法CSD，结合基于熵的概率松弛机制与最优重采样策略，提升自回归图像生成的推理效率。CSD利用图像不同区域的信息不确定性，动态调整候选令牌的接受概率，在低细节区域提高接受率以加速生成。此外，引入分布对齐过滤器，确保输出分布与目标模型一致，显著提升生成质量。在Lumina-mGPT和Janus-Pro上的实验表明，CSD具有明显优势。代码已开源：https://github.com/aderfebr/CSD。
原文摘要 · Abstract (English)
Speculative decoding (SD) has emerged as a key solution to accelerate the inference of autoregressive models. However, in the field of image generation, it faces the challenge of low acceptance rates, and directly relaxing its criteria leads to degradation in image quality. In this paper, we propose a novel content-aware speculative decoding algorithm, termed CSD, which integrates an entropy-based probability relaxation mechanism with an optimal resampling strategy to enhance the inference efficiency for autoregressive image generation. By leveraging the informational uncertainty inherent in different regions of an image, CSD dynamically adjusts the acceptance probability of candidate tokens, increasing the acceptance rate in low-detail areas to accelerate generation. Moreover, a distribution alignment filter is introduced to ensure the output distribution to be aligned with the target model, which significantly improves the generative quality. Experiments conducted on Lumina-mGPT and Janus-Pro demonstrate that the superiority of the proposed CSD. Our source code is available at https://github.com/aderfebr/CSD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。