用双码本机制在更小码本下实现高质量图像重建
Dual Codebook VQ: Enhanced Image Reconstruction with Reduced Codebook Size
- 将特征分为全局与局部两部分，分别用轻量Transformer和确定性选择处理
- 仅用512大小码本即达当前最佳重建效果，比之前方法减半
- 无需预训练，适合追求高效高质图像生成的研究者
向量量化（VQ）技术在码本利用上存在瓶颈，限制了图像建模的重建保真度。本文提出双码本机制，将表示分解为互补的全局与局部成分：全局码本采用轻量级Transformer实现所有码向量的同步更新，局部码本通过确定性选择保持精确特征表达。该方法从头训练，无需预训练知识。在多个标准基准数据集上的实验表明，本方法以512大小的紧凑码本实现了当前最优的重建质量，较需预训练的先前方法减少一半码本规模。在不同图像领域均取得显著的FID提升，尤其在场景与人脸重建任务中表现突出。结果证明，双码本VQ是一种高效且高保真的图像重建范式，大幅降低计算开销。
原文摘要 · Abstract (English)
Vector Quantization (VQ) techniques face significant challenges in codebook utilization, limiting reconstruction fidelity in image modeling. We introduce a Dual Codebook mechanism that effectively addresses this limitation by partitioning the representation into complementary global and local components. The global codebook employs a lightweight transformer for concurrent updates of all code vectors, while the local codebook maintains precise feature representation through deterministic selection. This complementary approach is trained from scratch without requiring pre-trained knowledge. Experimental evaluation across multiple standard benchmark datasets demonstrates state-of-the-art reconstruction quality while using a compact codebook of size 512 - half the size of previous methods that require pre-training. Our approach achieves significant FID improvements across diverse image domains, particularly excelling in scene and face reconstruction tasks. These results establish Dual Codebook VQ as an efficient paradigm for high-fidelity image reconstruction with significantly reduced computational requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。