用变分模型改进图像离散化，提升生成质量与稳定性。
VAEVQ: Enhancing Discrete Visual Tokenization through Variational Modeling
- 用变分自编码器替代传统架构，获得更平滑的潜在空间。
- 在两个基准数据集上优于现有方法，重建与生成性能提升。
- 适合需要高质量离散表征的生成模型研究者使用。
向量量化（VQ）将连续图像特征转换为离散表示，为生成模型提供压缩的标记输入。然而，基于VQ的框架存在潜在空间不平滑、量化前后表征对齐弱以及连续与离散域间一致性差等问题，导致码字学习不稳定、码本利用不足，进而降低重建和下游生成任务的性能。为此，我们提出VAEVQ，包含三个关键组件：(1) 变分潜在量化（VLQ），以变分自编码器（VAE）替代自编码器（AE）进行量化，利用其结构化且平滑的潜在空间，促进更有效的码字激活；(2) 表征一致性策略（RCS），自适应调节量化前后特征间的对齐强度，增强一致性并防止对噪声过拟合；(3) 分布一致性正则化（DCR），使整个码本分布与连续潜在分布对齐，提升码本利用率。在两个基准数据集上的大量实验表明，VAEVQ优于当前最先进方法。
原文摘要 · Abstract (English)
Vector quantization (VQ) transforms continuous image features into discrete representations, providing compressed, tokenized inputs for generative models. However, VQ-based frameworks suffer from several issues, such as non-smooth latent spaces, weak alignment between representations before and after quantization, and poor coherence between the continuous and discrete domains. These issues lead to unstable codeword learning and underutilized codebooks, ultimately degrading the performance of both reconstruction and downstream generation tasks. To this end, we propose VAEVQ, which comprises three key components: (1) Variational Latent Quantization (VLQ), replacing the AE with a VAE for quantization to leverage its structured and smooth latent space, thereby facilitating more effective codeword activation; (2) Representation Coherence Strategy (RCS), adaptively modulating the alignment strength between pre- and post-quantization features to enhance consistency and prevent overfitting to noise; and (3) Distribution Consistency Regularization (DCR), aligning the entire codebook distribution with the continuous latent distribution to improve utilization. Extensive experiments on two benchmark datasets demonstrate that VAEVQ outperforms state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。