提出动态调整树结构的采样方法，让图像自回归模型推理快3倍以上。
Fast Inference of Visual Autoregressive Model with Adjacency-Adaptive Dynamical Draft Trees
- 根据图像区域难易度动态调节采样树深度与宽度
- 在MS-COCO和PartiPrompts上分别提速3.13倍和3.05倍
- 适合追求高质图像生成速度的研究者与开发者
自回归图像模型虽能达到扩散模型的生成质量，但推理过程为串行，生成576x576图像需约2000步。尽管推测解码可加速语言模型，但在视觉自回归模型上表现不佳，因图像不同区域的标记预测难度差异大。本文识别出关键障碍：不同区域的接受率不一致。为此提出邻接自适应动态采样树（ADT-Tree），通过邻近标记状态与历史接受率动态调整树的深度与宽度。初始基于水平邻接构建，再通过二分适应优化，复杂区域更宽，简单区域更深。在MS-COCO 2017与PartiPrompts上的实验表明，该方法分别实现3.13倍与3.05倍加速。且可与宽松采样方法如LANTERN无缝结合，进一步提升效率。代码已开源。
原文摘要 · Abstract (English)
Autoregressive (AR) image models achieve diffusion-level quality but suffer from sequential inference, requiring approximately 2,000 steps for a 576x576 image. Speculative decoding with draft trees accelerates LLMs yet underperforms on visual AR models due to spatially varying token prediction difficulty. We identify a key obstacle in applying speculative decoding to visual AR models: inconsistent acceptance rates across draft trees due to varying prediction difficulties in different image regions. We propose Adjacency-Adaptive Dynamical Draft Trees (ADT-Tree), an adjacency-adaptive dynamic draft tree that dynamically adjusts draft tree depth and width by leveraging adjacent token states and prior acceptance rates. ADT-Tree initializes via horizontal adjacency, then refines depth/width via bisectional adaptation, yielding deeper trees in simple regions and wider trees in complex ones. The empirical evaluations on MS-COCO 2017 and PartiPrompts demonstrate that ADT-Tree achieves speedups of 3.13xand 3.05x, respectively. Moreover, it integrates seamlessly with relaxed sampling methods such as LANTERN, enabling further acceleration. Code is available at https://github.com/Haodong-Lei-Ray/ADT-Tree.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。