arXiv:2601.22244cs.CVcs.LG2026-01

单层量化也能达到与分层模型相当的重建质量。

Is Hierarchical Quantization Essential for Optimal Reconstruction?

  • 通过优化初始化和代码本管理,缓解单层量化中的崩溃问题。
  • 在相同表征预算下,单层模型重建精度可媲美分层模型。
  • 挑战了分层量化对高质量重建必不可少的普遍认知。

向量量化变分自编码器(VQ-VAE)是实现高重建保真度的核心技术,广泛应用于神经压缩与生成模型。尽管分层结构(如VQ-VAE2)被认为性能更优,因其能分离全局与局部特征,但高层信息完全依赖低层,不应携带额外重建内容。结合近期训练目标与量化机制的进步,我们质疑:在匹配表征预算且无代码本崩溃的前提下,单层VQ-VAE能否达到分层模型的重建精度?本文在高分辨率ImageNet图像上对比了两层与容量匹配的单层模型。结果表明,代码本利用不足会限制单层模型,而高维嵌入易引发量化不稳与代码本崩溃。通过数据初始化、周期性重置未激活代码本向量及系统调参等轻量干预,可显著减少崩溃。当表征预算匹配且代码本崩溃被抑制时,单层模型可达到与分层模型相当的重建精度,挑战了分层量化在高质量重建中固有优势的假设。

原文摘要 · Abstract (English)

Vector-quantized variational autoencoders (VQ-VAEs) are central to models that rely on high reconstruction fidelity, from neural compression to generative pipelines. Hierarchical extensions, such as VQ-VAE2, are often credited with superior reconstruction performance because they split global and local features across multiple levels. However, since higher levels derive all their information from lower levels, they should not carry additional reconstructive content beyond what the lower-level already encodes. Combined with recent advances in training objectives and quantization mechanisms, this leads us to ask whether a single-level VQ-VAE, with matched representational budget and no codebook collapse, can equal the reconstruction fidelity of its hierarchical counterpart. Although the multi-scale structure of hierarchical models may improve perceptual quality in downstream tasks, the effect of hierarchy on reconstruction accuracy, isolated from codebook utilization and overall representational capacity, remains empirically underexamined. We revisit this question by comparing a two-level VQ-VAE and a capacity-matched single-level model on high-resolution ImageNet images. Consistent with prior observations, we confirm that inadequate codebook utilization limits single-level VQ-VAEs and that overly high-dimensional embeddings destabilize quantization and increase codebook collapse. We show that lightweight interventions such as initialization from data, periodic reset of inactive codebook vectors, and systematic tuning of codebook hyperparameters significantly reduce collapse. Our results demonstrate that when representational budgets are matched, and codebook collapse is mitigated, single-level VQ-VAEs can match the reconstruction fidelity of hierarchical variants, challenging the assumption that hierarchical quantization is inherently superior for high-quality reconstructions.

VQ-VAE量化重建质量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。