arXiv:2605.06137cs.CVcs.AI2026-05

用前置提示符提升自回归图像生成质量,不损重建效果

Autoregressive Visual Generation Needs a Prologue

论文配图:Autoregressive Visual Generation Needs a Prologue
图 1 · 摘自论文原文
  • 引入前置提示符令牌,分离生成与重建任务
  • ImageNet上生成FID降至10.75,大型模型达rFID 0.99
  • 提示符自发形成语义结构,适合图像生成研究者

本文提出Prologue,解决自回归图像生成中重建与生成的矛盾。通过在视觉令牌序列前添加一组专用的前置提示符令牌,仅用AR交叉熵损失训练,而原视觉令牌保持重建职责不变。这种解耦设计使生成可直接优化于真实分布,同时不影响重建质量。在ImageNet 256x256上,Prologue-Base将gFID从21.01降至10.75,无需分类器自由引导;Prologue-Large以标准AR模型实现竞争性rFID 0.99和gFID 1.46,无辅助语义监督。有趣的是,仅由AR梯度驱动,提示符自发产生语义结构:对16个提示符线性探测达35.88% Top-1,远超标准分词器前16个令牌的23.71%;固定提示符重采样仍保留高层语义布局。结果表明,引入独立学习的生成表示可提升生成质量,同时保持原始表示完整。

原文摘要 · Abstract (English)

In this work, we propose Prologue, an approach to bridging the reconstruction-generation gap in autoregressive (AR) image generation. Instead of modifying visual tokens to satisfy both reconstruction and generation, Prologue generates a small set of prologue tokens prepended to the visual token sequence. These prologue tokens are trained exclusively with the AR cross-entropy (CE) loss, while visual tokens remain dedicated to reconstruction. This decoupled design lets us optimize generation through the AR model's true distribution without affecting reconstruction quality, which we further formalize from an ELBO perspective. On ImageNet 256x256, Prologue-Base reduces gFID from 21.01 to 10.75 without classifier-free guidance while keeping reconstruction almost unchanged; Prologue-Large reaches a competitive rFID of 0.99 and gFID of 1.46 using a standard AR model without auxiliary semantic supervision. Interestingly, driven only by AR gradients, prologue tokens exhibit emergent semantic structure: linear probing on 16 prologue tokens reaches 35.88% Top-1, far above the 23.71% of the first 16 tokens from a standard tokenizer; resampling with fixed prologue tokens preserves a similar high-level semantic layout. Our results suggest a new direction: generation quality can be improved by introducing a separate learned generative representation while leaving the original representation intact.

自回归生成图像生成提示符设计解耦优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。