随机打乱生成顺序，让图像模型更懂上下文关系。
Randomized Autoregressive Visual Generation
- 训练时随机打乱像素顺序，学习所有排列下的概率分布。
- ImageNet-256上FID达1.48，超越主流扩散与Transformer方法。
- 保持语言建模兼容性，适合想统一图文生成的团队使用。
本文提出随机自回归建模（RAR）用于视觉生成，在保持与语言建模范式完全兼容的同时，实现了图像生成任务的新状态。RAR方法简单：在标准自回归训练中，以概率r对通常按扫描线顺序排列的输入序列进行随机重排，r从1开始线性衰减至0。该渐进式训练策略使模型学会在所有因子化顺序上的期望似然，从而有效提升对双向上下文的建模能力。尤为重要的是，RAR保留了自回归框架的完整性，确保与语言建模的兼容性，同时显著提升图像生成性能。在ImageNet-256基准上，RAR达到1.48的FID分数，不仅超越此前最优的自回归图像生成器，还优于领先的扩散模型与掩码Transformer方法。代码与模型将公开于https://github.com/bytedance/1d-tokenizer。
原文摘要 · Abstract (English)
This paper presents Randomized AutoRegressive modeling (RAR) for visual generation, which sets a new state-of-the-art performance on the image generation task while maintaining full compatibility with language modeling frameworks. The proposed RAR is simple: during a standard autoregressive training process with a next-token prediction objective, the input sequence-typically ordered in raster form-is randomly permuted into different factorization orders with a probability r, where r starts at 1 and linearly decays to 0 over the course of training. This annealing training strategy enables the model to learn to maximize the expected likelihood over all factorization orders and thus effectively improve the model's capability of modeling bidirectional contexts. Importantly, RAR preserves the integrity of the autoregressive modeling framework, ensuring full compatibility with language modeling while significantly improving performance in image generation. On the ImageNet-256 benchmark, RAR achieves an FID score of 1.48, not only surpassing prior state-of-the-art autoregressive image generators but also outperforming leading diffusion-based and masked transformer-based methods. Code and models will be made available at https://github.com/bytedance/1d-tokenizer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。