代码本容量比分辨率更关键，决定视频压缩的感知质量。
Codebook Capacity Governs Perceptual Quality Across Resolutions in Hierarchical Discrete Video Compression
- 用不同代码本大小和分辨率组合实验，发现质量主要受代码本影响。
- 代码本容量每翻一倍，感知质量提升约0.94%，分辨率影响微不足道。
- 适合做多分辨率视频压缩系统或生成模型的离散编码器设计参考。
基于连续潜在表示的视频编解码器在扩展到新空间分辨率时，通常需要重新训练或调整率失真权衡，因为熵模型和拉格朗日权重与运行点紧密耦合。本文研究分层离散潜在编解码器是否具有相同敏感性。通过在UCF101数据集上对MS-VQ-VAE进行控制实验，测试了代码本大小K ∈ {128,256,512,1024}与分辨率64×64、128×128、256×256的组合，结果表明感知质量（LPIPS）强烈依赖代码本容量，而对空间分辨率影响极小。拟合对数线性模型Q(K,r) = αlog₂K + βlog₂r + γ于全部12个运行点，得到α = -0.0094（t = -6.6，p < 0.001），β = -0.0009（t = -0.43，p = 0.68，不显著），R² = 0.82。因此，代码本容量每增加一倍对感知质量的影响约为分辨率的10倍。同时，底层熵效率η = H(z)/log₂K在分辨率提高时保持稳定甚至提升（64×64时为84–87%，256×256时为92–94%），表明更大空间网格被更高效利用。所有分辨率与代码本组合下，模型在匹配或更低比特率时均优于H.264（128×128时提升25–52%），在256×256下优于H.265（提升21–37%）。结果表明，在分层离散视频编解码中，代码本大小K是主导感知压缩质量的设计变量，这一特性可简化多分辨率部署，并为生成式视频模型的可扩展离散分词器设计提供指导。
原文摘要 · Abstract (English)
Learned video codecs based on continuous latent representations typically require resolution-specific retraining or rate-distortion (RD) recalibration when scaling to new spatial resolutions, because entropy models and Lagrangian weights are tightly coupled to the operating point. We investigate whether hierarchical discrete latent codecs exhibit the same sensitivity. Using a controlled empirical study of MS-VQ-VAE video compression across codebook sizes $K \in \{128,256,512,1024\}$ and resolutions $64\times64$, $128\times128$, and $256\times256$ on UCF101, we show that perceptual quality (LPIPS) depends strongly on codebook capacity but only negligibly on spatial resolution. Fitting a log-linear model $Q(K,r) = α\log_2 K + β\log_2 r + γ$ to all 12 operating points yields $α=-0.0094$ ($t=-6.6$, $p<0.001$) and $β=-0.0009$ ($t=-0.43$, $p=0.68$, not significant), with $R^2=0.82$. Codebook capacity is therefore roughly $10\times$ more influential than spatial resolution per log-unit increase. In parallel, bottom-level entropy efficiency $η=H(z)/\log_2 K$ remains stable or improves with resolution (84-87% at $64\times64$; 92-94% at $256\times256$), confirming that larger spatial grids are utilized more efficiently rather than less. Across all resolutions and codebook sizes, our models outperform H.264 on LPIPS at matched or lower bitrate, with gains of 25-52% at $128\times128$ and 21-37% over H.265 at $256\times256$. These findings suggest that codebook size $K$, not spatial resolution, is the dominant design variable governing perceptual compression quality in hierarchical discrete video codecs -- a property that may simplify multi-resolution deployment and inform the design of scalable discrete tokenizers for generative video models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。