arXiv:2505.19089cs.CV2025-05NeurIPS被引 6

通过复用上下文特征,让图像生成模型更快且保持质量。

Plug-and-Play Context Feature Reuse for Efficient Masked Generation

  • 复用已解码的上下文特征,减少冗余计算。
  • 在ImageNet256上实现最高2.4倍加速,性能损失极小。
  • 可插拔设计,适配多种生成模型与架构。

掩码生成模型(MGMs)凭借并行解码与强双向上下文建模能力,在图像合成中表现优异。但高质量生成通常需大量迭代步骤,导致推理成本高。直接在每步解码更多标记虽可减少步骤数,却只能独立估计单变量分布,难以捕捉标记间依赖,显著降低生成质量。本文提出ReCAP(重用上下文感知预测),一种可插拔模块,通过复用先前解码的上下文特征嵌入,构建低成本迭代步骤。ReCAP交替执行标准全量评估与轻量级复用步骤,大幅降低计算量,同时保留细粒度迭代生成的优势。我们在三种代表性MGMs(MaskGIT、MAGE、MAR)上验证其有效性,涵盖离散与连续标记空间及多样架构。在ImageNet256类别条件生成任务中,ReCAP相比基线模型最快提升2.4倍推理速度,性能下降极小,并在不同生成设置下始终提供更优的效率-保真度权衡。

原文摘要 · Abstract (English)

Masked generative models (MGMs) have emerged as a powerful framework for image synthesis, combining parallel decoding with strong bidirectional context modeling. However, generating high-quality samples typically requires many iterative decoding steps, resulting in high inference costs. A straightforward way to speed up generation is by decoding more tokens in each step, thereby reducing the total number of steps. However, when many tokens are decoded simultaneously, the model can only estimate the univariate marginal distributions independently, failing to capture the dependency among them. As a result, reducing the number of steps significantly compromises generation fidelity. In this work, we introduce ReCAP (Reused Context-Aware Prediction), a plug-and-play module that accelerates inference in MGMs by constructing low-cost steps via reusing feature embeddings from previously decoded context tokens. ReCAP interleaves standard full evaluations with lightweight steps that cache and reuse context features, substantially reducing computation while preserving the benefits of fine-grained, iterative generation. We demonstrate its effectiveness on top of three representative MGMs (MaskGIT, MAGE, and MAR), including both discrete and continuous token spaces and covering diverse architectural designs. In particular, on ImageNet256 class-conditional generation, ReCAP achieves up to 2.4x faster inference than the base model with minimal performance drop, and consistently delivers better efficiency-fidelity trade-offs under various generation settings.

图像生成高效推理特征复用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。