32个离散令牌就能实现图像生成与编辑，无需训练
Highly Compressed Tokenizer Can Generate Without Training
- 用32个令牌压缩图像为一维序列，通过替换复制实现精细编辑
- 仅靠梯度优化令牌即可生成多样真实图像，无需训练模型
- 适合快速原型设计和无训练图像生成场景
常用图像分词器生成二维空间排列的令牌，而所谓的一维图像分词器将图像表示为仅含32个离散令牌的高度压缩一维序列。我们发现，这种使用向量量化实现的高压缩率使通过启发式操作令牌即可具备图像编辑与生成能力。即使粗略操作（如在不同图像潜在表示间复制和替换令牌），也能转移外观与语义属性，实现细粒度编辑。受一维分词器潜在空间表达力启发，我们构建了一个图像生成流程，利用基于梯度的测试时优化，在可插拔损失函数（如重建或CLIP相似性）下优化令牌。该方法在修补和文本引导图像编辑任务中得到验证，无需训练任何生成模型即可生成多样且真实的样本。
原文摘要 · Abstract (English)
Commonly used image tokenizers produce a 2D grid of spatially arranged tokens. In contrast, so-called 1D image tokenizers represent images as highly compressed one-dimensional sequences of as few as 32 discrete tokens. We find that the high degree of compression achieved by a 1D tokenizer with vector quantization enables image editing and generative capabilities through heuristic manipulation of tokens, demonstrating that even very crude manipulations -- such as copying and replacing tokens between latent representations of images -- enable fine-grained image editing by transferring appearance and semantic attributes. Motivated by the expressivity of the 1D tokenizer's latent space, we construct an image generation pipeline leveraging gradient-based test-time optimization of tokens with plug-and-play loss functions such as reconstruction or CLIP similarity. Our approach is demonstrated for inpainting and text-guided image editing use cases, and can generate diverse and realistic samples without requiring training of any generative model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。