用神经压缩模型指导传统编码器的比特分配,提升视觉质量。
Bit Allocation Transfer for Perceptual Quality Enhancement of Traditional Video Codecs

- 用神经网络学习帧内重要性,生成量化步长图。
- 在HEVC数据集上实现最高20.2%的MS-SSIM比特率节省。
- 无需改动编码器语法,可直接用于现有视频标准。
传统基于块的视频编码器(如H.264/AVC、H.265/HEVC和H.266/VVC)依赖手工设计的率失真优化(RDO)过程,主要最小化均方误差(MSE),与人类感知质量相关性较差。尽管神经视频压缩方法可轻松优化感知对齐指标(如MS-SSIM),但其高计算复杂度限制了实际部署。本文提出一种新型比特分配迁移框架,将神经视频压缩模型的隐式感知重要性迁移至传统编码器中,以提升其感知质量。具体而言,我们在神经视频压缩框架(DCVC-FM)中训练一个量化步长生成模型,输入为原始帧与运动补偿预测帧,输出为量化步长图。由此生成的块级比特比例被转换为传统编码器的量化参数(QP)图。在HEVC B~D数据集上的实验表明,相比标准参考软件JM-19.0、HM-16.20和VTM-23.0,本方法在MS-SSIM指标下分别实现了20.20%、8.25%和8.37%的码率节省,且使用预测帧时收益更显著。该方法在不修改传统编码器核心解码语法的前提下,有效将神经模型的感知优先级迁移至块级比特分配。
原文摘要 · Abstract (English)
Traditional block-based video codecs, such as H.264/AVC, H.265/HEVC and H.266/VVC, rely on hand-crafted Rate-Distortion Optimization (RDO) processes that primarily minimize Mean Squared Error (MSE), which correlates poorly with human perceptual quality. While neural video compression methods can easily optimize perceptually aligned metrics like MS-SSIM, their high computational complexity limits practical deployment. This paper proposes a novel bit allocation transfer framework that bridges these two paradigms to enhance the perceptual quality of conventional video codecs. Specifically, we train a quantization step generation model using a perceptual loss within a neural video compression framework (DCVC-FM). The model takes the original frame and a motion-compensated prediction as input and outputs a quantization step map. We then derive a block-wise bit ratio from this map and convert it into a Quantization Parameter (QP) map for a traditional video codec. Experimental results on the HEVC B$\sim$D dataset demonstrate that our method achieves 20.20\%, 8.25\%, and 8.37\% bitrate savings in terms of MS-SSIM compared with the standard reference software JM-19.0, HM-16.20, and VTM-23.0, respectively, with additional gains when utilizing predicted frames. Our approach effectively transfers the implicit perceptual importance learned by neural video compression models to guide block-level bit allocation in traditional video codecs without modifying their core decoding syntax.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。