让图像生成按需分配计算深度，提速三倍且画质损失小。
Depth Adaptive Efficient Visual Autoregressive Modeling
- 不删令牌，改按需分配每像素的计算深度。
- 实测速度提升2.3到3.1倍，画质几乎不变。
- 适合追求高效生成的视觉模型开发者。
视觉自回归（VAR）建模在生成高分辨率图像时对每个位置使用固定计算深度，效率低下。现有方法虽用频率图剪枝令牌加速推理，但其二值硬剪枝本质受限，即使频率估计更准也无法提升质量。观察到VAR模型存在显著深度冗余，我们提出从剪枝整令牌转向按令牌动态分配计算深度的范式转变。为此，提出无需训练的DepthVAR框架，通过循环旋转调度器动态分配深度，并以层为主导的掩码机制，在推理中选择性调用Transformer块，融合输出以确保每个令牌的影响与其处理深度成比例。大量实验表明，DepthVAR实现2.3×–3.1×加速，质量损失极小，相比现有硬剪枝方法具有更优的算力-性能权衡。代码已开源。
原文摘要 · Abstract (English)
Visual Autoregressive (VAR) modeling inefficiently applies a fixed computational depth to each position when generating high-resolution images. While existing methods accelerate inference by pruning tokens using frequency maps, their binary hard-pruning approach is fundamentally limited and fails to improve quality even with better frequency estimation. Observing that VAR models possess significant depth redundancy, we propose a paradigm shift from pruning entire tokens to adaptively allocating per-token computational depth. To this end, we introduce DepthVAR, a training-free framework that dynamically allocates computation. It integrates an adaptive depth scheduler, which assigns computational depth via a cyclic rotated schedule for balanced, non-static refinement, with a dynamic inference process that translates these depths into layer-major masks, selectively applies transformer blocks, and blends the resulting codes to ensure each token's influence is proportional to its processing depth. Extensive experiments show that DepthVAR achieves 2.3$\times$-3.1$\times$ acceleration with minimal quality loss, offering a competitive compute-performance trade-off compared to existing hard-pruning approaches. Code is available at https://github.com/STOVAGtz/DepthVAR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。