提出可学习的几何量化方法，让图像标记更稳定高效。
LGQ: Learnable Geometric Quantization for Image Tokenization
- 用可学习码本+温度退火实现软硬分配，无需固定网格
- 在65536码本下仍保持100%利用率，重建质量最优
- 适合需要高稳定性和高效率的图像生成任务
近期无崩溃量化器如FSQ通过固定标量网格替代可学习码本以实现稳定训练，但牺牲了灵活性。本文提出可学习几何量化（LGQ），保留可学习码本，并通过温度退火实现软到硬的分配，由两个低成本正则项驱动：按码本大小缩放的多样性项抑制批次平均使用集中，峰值项使每个标记的软分配趋向一热；二者协同防止码本崩溃，无需EMA、重置启发式或码本重参数化。在固定VQ-GAN主干下，我们在ImageNet 256x256上对比RotVQ、FSQ、SimVQ和IBQ，在K=16,384时，LGQ取得最佳重建FID且保持100%码本利用率；当代码本扩大至K=65,536时，利用率不变而性能持续提升。冻结分词器后训练MaskGIT，LGQ在条件生成方面优于其他量化器，重建与生成均领先。代码已开源。
原文摘要 · Abstract (English)
Recent collapse-free quantizers such as FSQ achieve stable training by replacing the learnable codebook with an engineered geometry: a fixed scalar grid whose structure is dictated by the codebook size K. We show this trade-off is unnecessary. We introduce Learnable Geometric Quantization (LGQ), which retains a learnable codebook of codes and performs soft-to-hard assignment via temperature annealing, regularized by two cheap terms: A diversity term scaled by codebook size that penalizes concentrated batch-average usage is the primary driver of collapse resistance, complemented by a peakedness term that sharpens each token's soft-assignment toward one-hot; together they prevent codebook collapse without EMA, reset heuristics, or codebook reparameterization. Under a fixed VQ-GAN backbone, we benchmark LGQ against RotVQ, FSQ, SimVQ, and IBQ on ImageNet 256x256 at K=16,384 and sweep LGQ over K={4096,..,65,536} without any per-K hyperparameter tuning. LGQ attains the best reconstruction FID at K = 16,384 while maintaining 100% codebook utilization, and continues to improve as the codebook grows to K = 65,536 with utilization unchanged. Training MaskGIT on the frozen tokenizers, LGQ further attains the best class-conditional generation among the compared quantizers, leading on reconstruction and generation alike. Code is available at https://github.com/KurbanIntelligenceLab/LGQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。