通过并行路径松弛采样,加速自回归文生图生成
PathRelax: Parallel-Path Relaxed Speculative Jacobi Decoding for Accelerating Auto-Regressive Text-to-Image Generation

- 构建多路径草稿树,扩大候选词搜索空间
- 在多个数据集上实现4倍以上推理加速,保持图像质量
- 可与其它加速技术融合,适合实时文生图场景
自回归文生图模型对高分辨率图像的需求导致序列变长,显著增加计算开销和推理时间。现有加速方法依赖链式草稿词序列,导致草稿搜索效率低、接受长度有限。为此,我们提出并行路径松弛推测雅可比解码框架(PathSpec),通过多序列草稿树结构提升效率。其中,并行路径推测雅可比解码(PathExplore)扩展了词元搜索空间,在不牺牲图像质量的前提下实现更高加速比。此外,跨路径松弛验证(PathRelax)利用序列间的语义相似性,进一步提升词元接受率。在Parti-Prompts、MSCOCO2017和T2ICompBench数据集上,分别获得4.14×、3.95×和4.18×的加速比。值得注意的是,PathExplore无需松弛采样即优于GSD和LANTERN等松弛方法。PathRelax的松弛机制可无缝集成其他技术,实现进一步加速,为实时文生图提供高效解决方案。
原文摘要 · Abstract (English)
The growing need for high-resolution image generation in autoregressive text-to-image models has resulted in extended token sequences, significantly increasing computational costs and inference times. However, existing state-of-the-art methods for accelerating autoregressive text-to-image models rely on chain-structured draft token sequences, leading to inefficient draft token search and limited acceptance lengths. To address this, we propose parallel-path cross-relaxed speculative Jacobi decoding (\textbf{PathSpec}), a novel framework that enhances efficiency through a multi-sequence draft tree structure. Our parallel-path speculative Jacobi decoding (\textbf{PathExplore}) expands the token search space, achieving a higher speedup ratio without sacrificing image quality. Additionally, we introduce cross-path relaxed verification (\textbf{PathRelax}) that exploits semantic similarities across sequences to further boost token acceptance rates. Evaluated on the Parti-Prompts, MSCOCO2017, and T2ICompBench datasets, our method achieves a speedup ratio of 4.14 $\times$, 3.95$\times$, and 4.18$\times$, respectively. Remarkably, PathExplore, without any relaxed sampling, outperforms relaxed sampling methods in the speedup ratio, such as GSD and LANTERN. Moreover, PathRelax's relaxation mechanism can be seamlessly integrated with other relaxation techniques, enabling further acceleration and providing an efficient solution for real-time text-to-image generation. Our code is available at https://github.com/Haodong-Lei-Ray/PathSpec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。