提出可折叠的语义分块令牌，提升图像自回归生成效率与质量。
ImageFolder: Autoregressive Image Generation with Folded Tokens
- 通过折叠机制让令牌在生成时更高效
- 用双分支量化捕获语义与细节，不增加令牌长度
- 适合追求高效高质图像生成的研究者
图像令牌化器对视觉生成模型（如扩散模型和自回归模型）至关重要，它构建了用于建模的潜在表示。增加令牌长度是提升图像重建质量的常见方法，但更长的令牌并不保证更好的生成质量，存在重建与生成质量之间的权衡。本文研究令牌长度对图像重建和生成的影响，并提出灵活解决方案——ImageFolder，一种空间对齐的语义令牌化器，可在自回归建模中折叠令牌以提升生成效率与质量。为在不增加令牌长度的前提下增强表征能力，采用双分支产品量化：一个分支引入语义正则化以压缩语义信息，另一分支捕捉剩余像素级细节。大量实验表明，ImageFolder在更短令牌长度下实现更优的图像生成质量。
原文摘要 · Abstract (English)
Image tokenizers are crucial for visual generative models, e.g., diffusion models (DMs) and autoregressive (AR) models, as they construct the latent representation for modeling. Increasing token length is a common approach to improve the image reconstruction quality. However, tokenizers with longer token lengths are not guaranteed to achieve better generation quality. There exists a trade-off between reconstruction and generation quality regarding token length. In this paper, we investigate the impact of token length on both image reconstruction and generation and provide a flexible solution to the tradeoff. We propose ImageFolder, a semantic tokenizer that provides spatially aligned image tokens that can be folded during autoregressive modeling to improve both generation efficiency and quality. To enhance the representative capability without increasing token length, we leverage dual-branch product quantization to capture different contexts of images. Specifically, semantic regularization is introduced in one branch to encourage compacted semantic information while another branch is designed to capture the remaining pixel-level details. Extensive experiments demonstrate the superior quality of image generation and shorter token length with ImageFolder tokenizer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。