多组量化让离散编码器重建更准，超越传统VAE
MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
- 用多组子码本保留特征维度，提升离散编码表示能力
- 在ImageNet上rFID达0.49，显著优于SD-VAE的0.91
- 适用于高清图像保真任务，适合追求高质量重建的研究者
向量量化变分自编码器（VQ-VAEs）是将连续视觉数据压缩为离散标记的基础模型。现有方法虽尝试优化量化策略以提升重建质量，但VQ-VAEs与VAEs之间仍存在显著差距。为此，本文提出MGVQ，通过保留潜在维度并引入多组子码本增强代码本表征能力，使码本优化更易实现且信息损失更小，从而提升重建质量。此外，构建了包含512p和2k分辨率的全面零样本基准测试集，严格评估现有方法性能。MGVQ在ImageNet及8个零样本基准上均达到当前最优表现。相较于SD-VAE，其在ImageNet上的rFID从0.91降至0.49，所有零样本基准的PSNR也全面领先。结果表明MGVQ在重建精度上的优势，为高分辨率图像处理中的保真性提供了新路径。代码将公开于https://github.com/MKJia/MGVQ。
原文摘要 · Abstract (English)
Vector Quantized Variational Autoencoders (VQ-VAEs) are fundamental models that compress continuous visual data into discrete tokens. Existing methods have tried to improve the quantization strategy for better reconstruction quality, however, there still exists a large gap between VQ-VAEs and VAEs. To narrow this gap, we propose MGVQ, a novel method to augment the representation capability of discrete codebooks, facilitating easier optimization for codebooks and minimizing information loss, thereby enhancing reconstruction quality. Specifically, we propose to retain the latent dimension to preserve encoded features and incorporate a set of sub-codebooks for quantization. Furthermore, we construct comprehensive zero-shot benchmarks featuring resolutions of 512p and 2k to evaluate the reconstruction performance of existing methods rigorously. MGVQ achieves the state-of-the-art performance on both ImageNet and 8 zero-shot benchmarks across all VQ-VAEs. Notably, compared with SD-VAE, we outperform them on ImageNet significantly, with rFID 0.49 v.s. 0.91, and achieve superior PSNR on all zero-shot benchmarks. These results highlight the superiority of MGVQ in reconstruction and pave the way for preserving fidelity in HD image processing tasks. Code will be publicly available at https://github.com/MKJia/MGVQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。