arXiv:2506.08908cs.CV2025-06被引 11

通过自适应跳过冗余步骤,实现视觉自回归生成的高效加速。

SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skipping

  • 基于频率感知动态选择跳过策略,减少生成冗余步骤。
  • 在GenEval上实现最高2.62倍加速,平均SSIM仍超0.88。
  • 无需训练,适配不同样本,适合高效率图像生成场景。

视觉自回归(VAR)模型的研究表明,生成过程中的高频成分或后期步骤对推理延迟贡献显著。然而,这些步骤背后的计算冗余尚未被充分探讨。本文深入分析了VAR推理过程,发现两大效率瓶颈:步骤冗余与无条件分支冗余。针对步骤冗余,提出自动跳步策略,有选择地省略不必要的生成步骤以提升效率;针对无条件分支冗余,发现条件与无条件分支间信息差距极小,因此引入无条件分支替换技术,直接跳过该分支以降低计算开销。值得注意的是,加速策略的效果在不同样本间差异显著,为此提出样本自适应框架SkipVAR,利用频率信息动态选择最优加速策略。为评估高频信息的作用,构建高变化性基准数据集,测试模型对细节的敏感性。大量实验表明,SkipVAR在保持模型质量的前提下,实现平均0.88以上SSIM,整体加速比达1.81倍,GenEval上最高提速2.62倍。结果验证了频率感知、免训练自适应加速在可扩展自回归图像生成中的有效性。代码已开源。

原文摘要 · Abstract (English)

Recent studies on Visual Autoregressive (VAR) models have highlighted that high-frequency components, or later steps, in the generation process contribute disproportionately to inference latency. However, the underlying computational redundancy involved in these steps has yet to be thoroughly investigated. In this paper, we conduct an in-depth analysis of the VAR inference process and identify two primary sources of inefficiency: step redundancy and unconditional branch redundancy. To address step redundancy, we propose an automatic step-skipping strategy that selectively omits unnecessary generation steps to improve efficiency. For unconditional branch redundancy, we observe that the information gap between the conditional and unconditional branches is minimal. Leveraging this insight, we introduce unconditional branch replacement, a technique that bypasses the unconditional branch to reduce computational cost. Notably, we observe that the effectiveness of acceleration strategies varies significantly across different samples. Motivated by this, we propose SkipVAR, a sample-adaptive framework that leverages frequency information to dynamically select the most suitable acceleration strategy for each instance. To evaluate the role of high-frequency information, we introduce high-variation benchmark datasets that test model sensitivity to fine details. Extensive experiments show SkipVAR achieves over 0.88 average SSIM with up to 1.81x overall acceleration and 2.62x speedup on the GenEval benchmark, maintaining model quality. These results confirm the effectiveness of frequency-aware, training-free adaptive acceleration for scalable autoregressive image generation. Our code is available at https://github.com/fakerone-li/SkipVAR and has been publicly released.

自回归生成加速推理频率感知图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。