arXiv:2608.25386cs.CV2026-08中稿 · ACM Multimedia 202…

提升自回归图像生成效率与质量,通过多标记预测和语义丢弃实现更快训练。

Efficient Training with Foresight: Multi-Token Auxiliary Supervision for Autoregressive Image Generation

论文配图:Efficient Training with Foresight: Multi-Token Auxiliary Supervision for Autoregressive Image Generation
图 1 · 摘自论文原文
  • 引入多标记预测,联合监督多个未来标记,缓解传统方法的稀疏与短视问题。
  • 在ImageNet上训练速度提升39%,FID降低0.95,仅用1/3迭代次数即达更好性能。
  • 适合追求高效高质图像生成的开发者,尤其关注训练成本优化的研究者。

自回归(AR)图像生成通过将图像建模为离散标记序列,在可扩展性与高保真合成方面展现出强大潜力。然而,传统的下一步标记预测(NTP)仍面临监督稀疏、目光短浅、表征判别性不足以及全序列密集计算带来的高训练成本等问题。为此,我们提出多标记自回归(MTAR),一个统一的训练框架,从预测目标、表征正则化和训练效率三方面改进自回归图像生成。具体而言,MTAR引入多标记预测(MTP),通过联合监督多个未来标记缓解传统NTP的稀疏性和短视性;采用标记级对比正则化(TCR),显式增强采样标记表示的可分性,提升表征判别能力;并引入语义丢弃(SD)作为语义感知的训练加速策略,减少低信息标记的冗余计算,同时保留有效学习信号。三项组件均仅用于训练,推理阶段无额外开销。在ImageNet上,MTAR实现了生成质量与训练效率的更优平衡。相比LlamaGen,MTAR实现最高0.95的FID降低和39%的训练加速。即使仅使用1/3的训练迭代次数,其性能仍可达到或优于基线,显著缩短训练时间。

原文摘要 · Abstract (English)

Autoregressive (AR) image generation has shown strong potential for scalable high-fidelity synthesis by modeling images as discrete token sequences. However, traditional next token prediction (NTP) continues to suffer from sparse and myopic supervision, insufficiently discriminative representations, and high training cost caused by dense computation over the full token sequence. To address these issues, we propose multi-token autoregressive (MTAR), a unified training framework that improves autoregressive image generation from three aspects: prediction objectives, representation regularization, and training efficiency. Specifically, MTAR introduces multi-token prediction (MTP) to alleviate the sparsity and myopia of traditional NTP by imposing joint supervision on multiple future tokens; employs token-level contrastive regularization (TCR) to explicitly enhance the separability of sampled token representations and thereby improve representation discriminability; and incorporates semantic dropping (SD) as a semantics-aware training acceleration strategy to reduce redundant computation on low-information tokens while preserving informative learning signals. All three components are applied only during training and introduce no additional overhead during autoregressive inference. On ImageNet, MTAR achieves a better balance between generation quality and training efficiency. Compared with LlamaGen, MTAR achieves up to 0.95 lower FID and 39\% faster training. Moreover, even with only 1/3 of the training iterations, it still attains performance comparable to or better than the baseline, substantially reducing training time.

自回归生成图像生成训练效率多标记预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。