用二值令牌生成图像，速度和质量均领先现有自回归模型。
BitDance: Scaling Autoregressive Generative Models with Binary Tokens
- 用二值潜变量替代代码本索引，每令牌可表示2^256种状态。
- 在ImageNet上达FID 1.24，参数量仅260M却快8.7倍。
- 适合追求高效高质图像生成的开发者与研究者。
我们提出BitDance，一种可扩展的自回归（AR）图像生成模型，通过预测二值视觉令牌而非代码本索引实现高效生成。高熵二值潜变量使每个令牌最多表示$2^{256}$种状态，形成紧凑而高度表达的离散表征。标准分类难以采样如此庞大的令牌空间，为此BitDance引入二值扩散头：不使用softmax预测索引，而是采用连续空间扩散生成二值令牌。此外，我们提出新解码方法——下一补丁扩散（next-patch diffusion），可并行高精度预测多个令牌，大幅加速推理。在ImageNet 256x256上，BitDance达到FID 1.24，为当前自回归模型最优表现。结合next-patch diffusion，其以260M参数量超越使用1.4B参数的顶尖并行自回归模型，实现8.7倍加速。在文本到图像生成任务中，模型基于大规模多模态令牌训练，能高效生成高分辨率、逼真图像，展现良好扩展性。生成1024x1024图像时，较先前自回归模型提速超30倍。代码与模型已开源，以促进自回归基础模型研究。代码与模型地址：https://github.com/shallowdream204/BitDance。
原文摘要 · Abstract (English)
We present BitDance, a scalable autoregressive (AR) image generator that predicts binary visual tokens instead of codebook indices. With high-entropy binary latents, BitDance lets each token represent up to $2^{256}$ states, yielding a compact yet highly expressive discrete representation. Sampling from such a huge token space is difficult with standard classification. To resolve this, BitDance uses a binary diffusion head: instead of predicting an index with softmax, it employs continuous-space diffusion to generate the binary tokens. Furthermore, we propose next-patch diffusion, a new decoding method that predicts multiple tokens in parallel with high accuracy, greatly speeding up inference. On ImageNet 256x256, BitDance achieves an FID of 1.24, the best among AR models. With next-patch diffusion, BitDance beats state-of-the-art parallel AR models that use 1.4B parameters, while using 5.4x fewer parameters (260M) and achieving 8.7x speedup. For text-to-image generation, BitDance trains on large-scale multimodal tokens and generates high-resolution, photorealistic images efficiently, showing strong performance and favorable scaling. When generating 1024x1024 images, BitDance achieves a speedup of over 30x compared to prior AR models. We release code and models to facilitate further research on AR foundation models. Code and models are available at: https://github.com/shallowdream204/BitDance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。