arXiv:2412.14170cs.CVcs.AI2024-12被引 5

E-CAR通过分阶段生成连续令牌,实现高效图像生成。

E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling

  • 分阶段生成连续令牌,逐步细化图像细节
  • 相比传统方法减少10倍计算量,速度提升5倍
  • 适合追求高效生成的视觉模型研究者

连续令牌的自回归模型在图像生成中展现出良好前景,无需离散化。但其序列生成和依赖高成本扩散采样带来效率问题。本文提出E-CAR(基于多阶段建模的高效连续自回归图像生成),通过两项创新:(1) 分阶段连续令牌生成策略,降低计算复杂度并提供分层条件;(2) 多阶段流模型,仅对部分去噪分布进行变换,而非完整去噪。整体上,E-CAR在逐级提高分辨率的同时进行图像去噪。该设计将令牌到图像转换成本降低为阶段数倍,支持令牌级并行处理。实验表明,E-CAR在生成256×256图像时,图像质量与DiT Peebles & Xie [2023]相当,但所需计算量减少10倍(FLOPs),速度提升5倍。

原文摘要 · Abstract (English)

Recent advances in autoregressive (AR) models with continuous tokens for image generation show promising results by eliminating the need for discrete tokenization. However, these models face efficiency challenges due to their sequential token generation nature and reliance on computationally intensive diffusion-based sampling. We present ECAR (Efficient Continuous Auto-Regressive Image Generation via Multistage Modeling), an approach that addresses these limitations through two intertwined innovations: (1) a stage-wise continuous token generation strategy that reduces computational complexity and provides progressively refined token maps as hierarchical conditions, and (2) a multistage flow-based distribution modeling method that transforms only partial-denoised distributions at each stage comparing to complete denoising in normal diffusion models. Holistically, ECAR operates by generating tokens at increasing resolutions while simultaneously denoising the image at each stage. This design not only reduces token-to-image transformation cost by a factor of the stage number but also enables parallel processing at the token level. Our approach not only enhances computational efficiency but also aligns naturally with image generation principles by operating in continuous token space and following a hierarchical generation process from coarse to fine details. Experimental results demonstrate that ECAR achieves comparable image quality to DiT Peebles & Xie [2023] while requiring 10$\times$ FLOPs reduction and 5$\times$ speedup to generate a 256$\times$256 image.

图像生成自回归模型高效生成连续令牌

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。