用无限词汇位元建模生成高清逼真图像，速度超扩散模型2.6倍。
Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis
- 采用位元级预测与无限词表，提升图像生成细节与能力
- 1024x1024图像生成仅需0.8秒，比SD3-Medium快2.6倍
- 无需额外优化即可超越主流扩散模型，适合高效图像生成场景
我们提出Infinity，一种基于位元级自回归建模的视觉生成方法，可依据文本指令生成高分辨率、逼真图像。Infinity在位元级令牌预测框架下，引入无限词汇量分词器与分类器，并结合位元级自我修正机制，显著提升生成能力和细节表现。通过理论将分词器词汇量扩展至无穷大，并同步扩大Transformer规模，其拓展能力远超传统自回归模型。在GenEval基准上，分数从0.62提升至0.73；ImageReward得分从0.87升至0.96，胜率66%。无需额外优化，1024x1024图像生成耗时仅0.8秒，比SD3-Medium快2.6倍，成为当前最快的文本到图像模型。相关模型与代码将开源，推动视觉生成与统一分词建模研究。
原文摘要 · Abstract (English)
We present Infinity, a Bitwise Visual AutoRegressive Modeling capable of generating high-resolution, photorealistic images following language instruction. Infinity redefines visual autoregressive model under a bitwise token prediction framework with an infinite-vocabulary tokenizer & classifier and bitwise self-correction mechanism, remarkably improving the generation capacity and details. By theoretically scaling the tokenizer vocabulary size to infinity and concurrently scaling the transformer size, our method significantly unleashes powerful scaling capabilities compared to vanilla VAR. Infinity sets a new record for autoregressive text-to-image models, outperforming top-tier diffusion models like SD3-Medium and SDXL. Notably, Infinity surpasses SD3-Medium by improving the GenEval benchmark score from 0.62 to 0.73 and the ImageReward benchmark score from 0.87 to 0.96, achieving a win rate of 66%. Without extra optimization, Infinity generates a high-quality 1024x1024 image in 0.8 seconds, making it 2.6x faster than SD3-Medium and establishing it as the fastest text-to-image model. Models and codes will be released to promote further exploration of Infinity for visual generation and unified tokenizer modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。