通过分两阶段生成图像,实现掩码自回归模型的3.72倍加速。
Generation then Reconstruction: Accelerating Masked Autoregressive Models via Two-Stage Sampling
- 先生成图像结构再重建细节,分步降低计算复杂度。
- 在ImageNet上实现3.72倍加速,FID仅1.59,质量接近原模型。
- 适合需要快速生成高质量图像的应用场景。
掩码自回归(MAR)模型虽具备并行生成优势,但受限于视觉标记的空间相关性建模复杂度。本文提出无需训练的两阶段采样策略GtR:先生成全局语义结构,再高效补全细节。基于‘补全比从零生成更易’的假设,慢速计算生成阶段以保质量,快速完成重建阶段以提速。同时,针对图像细节区域携带更多语义信息的现象,提出基于高频能量的频率加权标记选择(FTS),将更多计算资源分配至细节区域。在ImageNet条件生成与文生图任务中,MAR-H模型实现3.72倍加速,保持优异质量(如FID: 1.59, IS: 304.4),显著优于现有加速方法,适用于多种模型规模与任务。代码将在https://github.com/feihongyan1/GtR发布。
原文摘要 · Abstract (English)
Masked Autoregressive (MAR) models promise better efficiency in visual generation than autoregressive (AR) models for the ability of parallel generation, yet their acceleration potential remains constrained by the modeling complexity of spatially correlated visual tokens in a single step. To address this limitation, we introduce Generation then Reconstruction (GtR), a training-free hierarchical sampling strategy that decomposes generation into two stages: structure generation establishing global semantic scaffolding, followed by detail reconstruction efficiently completing remaining tokens. Assuming that it is more difficult to create an image from scratch than to complement images based on a basic image framework, GtR is designed to achieve acceleration by computing the reconstruction stage quickly while maintaining the generation quality by computing the generation stage slowly. Moreover, observing that tokens on the details of an image often carry more semantic information than tokens in the salient regions, we further propose Frequency-Weighted Token Selection (FTS) to offer more computation budget to tokens on image details, which are localized based on the energy of high frequency information. Extensive experiments on ImageNet class-conditional and text-to-image generation demonstrate 3.72x speedup on MAR-H while maintaining comparable quality (e.g., FID: 1.59, IS: 304.4 vs. original 1.59, 299.1), substantially outperforming existing acceleration methods across various model scales and generation tasks. Our codes will be released in https://github.com/feihongyan1/GtR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。