开源复现超大词典视觉生成模型，推动自回归图像生成民主化
Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generation
- 用非对称分块策略处理超大词典，提升自回归生成效率
- 在ImageNet上重建性能达当前最优，零样本指标1.93优于同类模型
- 提供300M到1.5B参数的系列模型，适合研究与应用开发者
Open-MAGVIT2项目开源复现了Google的MAGVIT-v2分词器，该分词器拥有超大词典（$2^{18}$个码本），在ImageNet和UCF基准上实现当前最佳重建性能。我们还提供了在大规模数据上预训练的分词器，在零样本任务中显著超越Cosmos（ImageNet原分辨率rFID：1.93 vs. 0.78）。进一步探索其在纯自回归模型中的应用，构建了从300M到1.5B参数的系列生成模型。为解决超大词汇量下的预测难题，采用非对称分词因子分解策略，并引入“下一子词预测”机制以增强子词间交互，提升生成质量。所有模型与代码均已公开，旨在推动自回归视觉生成领域的创新。
原文摘要 · Abstract (English)
The Open-MAGVIT2 project produces an open-source replication of Google's MAGVIT-v2 tokenizer, a tokenizer with a super-large codebook (i.e., $2^{18}$ codes), and achieves the state-of-the-art reconstruction performance on ImageNet and UCF benchmarks. We also provide a tokenizer pre-trained on large-scale data, significantly outperforming Cosmos on zero-shot benchmarks (1.93 vs. 0.78 rFID on ImageNet original resolution). Furthermore, we explore its application in plain auto-regressive models to validate scalability properties, producing a family of auto-regressive image generation models ranging from 300M to 1.5B. To assist auto-regressive models in predicting with a super-large vocabulary, we factorize it into two sub-vocabulary of different sizes by asymmetric token factorization, and further introduce ``next sub-token prediction'' to enhance sub-token interaction for better generation quality. We release all models and codes to foster innovation and creativity in the field of auto-regressive visual generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。