端到端训练让图像生成模型直接优化分词器，提升生成质量。
End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer
- 联合优化分词器与生成模型，实现端到端训练
- 在ImageNet 256x256上达1.48的FID，为当前最优
- 利用视觉基础模型增强1D分词器，适合生成任务研究者
自回归图像建模依赖视觉分词器将图像压缩为紧凑的潜在表示。本文设计了一个端到端训练流程，联合优化重建与生成，使生成结果能直接监督分词器。这与以往先训练分词器再训练生成模型的两阶段方法形成对比。此外，我们探索了利用视觉基础模型改进用于自回归建模的1D分词器。所提出的自回归生成模型取得了强劲的实证效果，在无引导条件下于ImageNet 256x256生成任务上达到1.48的最新版FID分数。
原文摘要 · Abstract (English)
Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256x256 generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。