用二进制比特令牌直接生成图像，性能超越扩散模型。
MaskBit: Embedding-free Image Generation via Bit Tokens
- 抛弃嵌入层，直接在二进制比特上操作生成图像。
- 在ImageNet 256x256上达到1.52的FID，参数仅305M。
- 适合关注高效图像生成与模型轻量化的研究者。
用于类别条件图像生成的掩码Transformer模型已成为扩散模型的有力替代方案。这类框架通常包含两个阶段：首先使用VQGAN在潜在空间与图像空间之间转换，随后通过Transformer在潜在空间中生成图像。本文提出两项主要贡献：第一，对VQGAN进行系统性实证研究，构建出现代化的高性能VQGAN模型，具备可复现性与高精度，性能媲美当前顶尖方法，并揭示了此前未被发现的细节；第二，提出一种无需嵌入的生成网络，直接基于具有丰富语义的二进制量化比特令牌进行图像生成。该方法在ImageNet 256x256基准上实现了1.52的FID新纪录，且生成器模型仅含305M参数。项目代码已公开于https://github.com/markweberdev/maskbit。
原文摘要 · Abstract (English)
Masked transformer models for class-conditional image generation have become a compelling alternative to diffusion models. Typically comprising two stages - an initial VQGAN model for transitioning between latent space and image space, and a subsequent Transformer model for image generation within latent space - these frameworks offer promising avenues for image synthesis. In this study, we present two primary contributions: Firstly, an empirical and systematic examination of VQGANs, leading to a modernized VQGAN. Secondly, a novel embedding-free generation network operating directly on bit tokens - a binary quantized representation of tokens with rich semantics. The first contribution furnishes a transparent, reproducible, and high-performing VQGAN model, enhancing accessibility and matching the performance of current state-of-the-art methods while revealing previously undisclosed details. The second contribution demonstrates that embedding-free image generation using bit tokens achieves a new state-of-the-art FID of 1.52 on the ImageNet 256x256 benchmark, with a compact generator model of mere 305M parameters. The code for this project is available on https://github.com/markweberdev/maskbit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。