arXiv:2412.15119cs.CV2024-12CVPR被引 61

让视觉生成更快:通过并行处理弱依赖图像块,速度提升9.5倍。

Parallelized Autoregressive Visual Generation

  • 按视觉标记依赖强度分组,弱依赖项并行生成,强依赖项仍串行处理。
  • ImageNet上速度提升3.6倍,视频任务最高提速9.5倍,质量几乎不变。
  • 无需改动模型结构,可直接用于现有自回归生成模型。

自回归模型在视觉生成中表现强大,但因逐标记生成导致推理缓慢。本文提出一种简单有效的并行化自回归视觉生成方法,在保持自回归优势的同时显著提升效率。核心思路是:视觉标记间存在依赖差异——弱依赖的标记可并行生成,而强依赖的相邻标记若独立采样易引发不一致。据此,我们设计策略:对远距离弱依赖标记并行生成,对局部强依赖标记保持串行生成。该方法无需修改模型架构或分词器,可无缝集成至标准自回归模型。在ImageNet和UCF-101上的实验表明,该方法在图像和视频生成任务中分别实现3.6倍和高达9.5倍的速度提升,且质量损失极小。我们希望本工作能推动高效视觉生成与统一自回归建模的研究。

原文摘要 · Abstract (English)

Autoregressive models have emerged as a powerful approach for visual generation but suffer from slow inference speed due to their sequential token-by-token prediction process. In this paper, we propose a simple yet effective approach for parallelized autoregressive visual generation that improves generation efficiency while preserving the advantages of autoregressive modeling. Our key insight is that parallel generation depends on visual token dependencies-tokens with weak dependencies can be generated in parallel, while strongly dependent adjacent tokens are difficult to generate together, as their independent sampling may lead to inconsistencies. Based on this observation, we develop a parallel generation strategy that generates distant tokens with weak dependencies in parallel while maintaining sequential generation for strongly dependent local tokens. Our approach can be seamlessly integrated into standard autoregressive models without modifying the architecture or tokenizer. Experiments on ImageNet and UCF-101 demonstrate that our method achieves a 3.6x speedup with comparable quality and up to 9.5x speedup with minimal quality degradation across both image and video generation tasks. We hope this work will inspire future research in efficient visual generation and unified autoregressive modeling. Project page: https://yuqingwang1029.github.io/PAR-project.

图像生成自回归加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。