让分词器直接学习去噪，提升图像生成质量
Latent Denoising Makes Good Tokenizers
- 分词器嵌入对齐下游去噪目标，增强鲁棒性
- 在六种模型上均显著提升生成质量
- 适合关注分词器设计与生成模型优化的研究者
尽管分词器在生成建模中起着基础作用，但其有效性依赖的特性仍不明确。我们观察到现代生成模型共享一种相似的训练目标——从受扰输入中重建干净信号，如高斯噪声或掩码造成的退化，这一过程称为去噪。受此启发，我们提出将分词器嵌入直接对齐下游去噪目标，使潜在表示在严重干扰下仍可重建。为此，我们引入了潜空间去噪分词器（l-DeTok），一种通过插值噪声或随机掩码对潜在嵌入进行扰动并重建干净图像来训练的简单而高效的分词器。在类条件（ImageNet 256x256 和 512x512）和文本条件（MSCOCO）图像生成基准上的大量实验表明，与以往分词器相比，l-DeTok 在六种代表性生成模型上均持续提升了生成质量。研究结果强调去噪是分词器设计的基本原则，希望为未来分词器设计提供新视角。
原文摘要 · Abstract (English)
Despite their fundamental role, it remains unclear what properties could make tokenizers more effective for generative modeling. We observe that modern generative models share a conceptually similar training objective -- reconstructing clean signals from corrupted inputs, such as signals degraded by Gaussian noise or masking -- a process we term denoising. Motivated by this insight, we propose aligning tokenizer embeddings directly with the downstream denoising objective, encouraging latent embeddings that remain reconstructable even under significant corruption. To achieve this, we introduce the Latent Denoising Tokenizer (l-DeTok), a simple yet highly effective tokenizer trained to reconstruct clean images from latent embeddings corrupted via interpolative noise or random masking. Extensive experiments on class-conditioned (ImageNet 256x256 and 512x512) and text-conditioned (MSCOCO) image generation benchmarks demonstrate that our l-DeTok consistently improves generation quality across six representative generative models compared to prior tokenizers. Our findings highlight denoising as a fundamental design principle for tokenizer development, and we hope it could motivate new perspectives for future tokenizer design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。