提出缓存机制，让图像生成模型加速近3倍且质量不变。
LazyMAR: Accelerating Masked Autoregressive Models via Feature Caching
- 利用相邻步骤中令牌和条件输出的冗余，分两步缓存复用。
- 在多个MAR模型上实现2.83倍加速，生成质量几乎无损。
- 无需训练、可即插即用，适合追求高效生成的开发者。
掩码自回归（MAR）模型在图像生成中展现出潜力，理论上可通过并行解码提升计算效率。然而，其对双向自注意力的依赖与传统键值（KV）缓存机制冲突，导致意外的计算瓶颈，削弱了预期效率。本文研究了MAR的缓存机制，发现两类冗余：一是相邻解码步骤中大量令牌表示高度相似，可提前缓存并复用；二是分类器无关引导下条件与无条件输出在相邻步骤中差异趋同。基于此，提出LazyMAR，引入两种缓存机制分别处理上述冗余。该方法无需训练，适用于所有MAR模型。实验表明，其在保持生成质量几乎不变的前提下，实现了2.83倍加速。代码将发布于https://github.com/feihongyan1/LazyMAR。
原文摘要 · Abstract (English)
Masked Autoregressive (MAR) models have emerged as a promising approach in image generation, expected to surpass traditional autoregressive models in computational efficiency by leveraging the capability of parallel decoding. However, their dependence on bidirectional self-attention inherently conflicts with conventional KV caching mechanisms, creating unexpected computational bottlenecks that undermine their expected efficiency. To address this problem, this paper studies the caching mechanism for MAR by leveraging two types of redundancy: Token Redundancy indicates that a large portion of tokens have very similar representations in the adjacent decoding steps, which allows us to first cache them in previous steps and then reuse them in the later steps. Condition Redundancy indicates that the difference between conditional and unconditional output in classifier-free guidance exhibits very similar values in adjacent steps. Based on these two redundancies, we propose LazyMAR, which introduces two caching mechanisms to handle them one by one. LazyMAR is training-free and plug-and-play for all MAR models. Experimental results demonstrate that our method achieves 2.83 times acceleration with almost no drop in generation quality. Our codes will be released in https://github.com/feihongyan1/LazyMAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。