首次实现神经视频压缩的像素级连续局部码率控制，提升质量分布灵活性。
LRC-DHVC: Towards Local Rate Control in Neural Video Compression
- 通过拼接质量图并使用加权MSE损失，实现像素级质量调控。
- 单套参数支持宽范围码率，内存开销恒定，优于多速率点模型。
- 适合需要动态质量分配的机器视觉视频编码场景。
局部码率控制是推动图像与视频压缩适应特定挑战（如面向机器的视频编码）的关键。传统混合视频编码可通过调整局部量化参数灵活调节局部率失真权衡，但现有基于学习的视频压缩方法尚无类似机制。本文提出LRC-DHVC，一种分层视频压缩网络，可在像素级别实现连续的局部码率控制，以调整单帧内空间质量分布。该方法通过将质量图拼接到输入帧，并应用与质量图中像素级权衡因子匹配的加权均方误差损失来实现。训练时，模型通过受限随机生成多种质量图进行学习。本模型是首个能连续、空间化适应不同质量约束的神经视频压缩网络。由于覆盖广泛的质量与码率范围，仅需一组网络参数即可满足需求，内存开销不随速率点数量增加而增长。相比线性扩展的单速率点网络，显著降低存储成本。代码与模型将在录用后公开。
原文摘要 · Abstract (English)
Local rate control is a key enabler to generalize image and video compression for dedicated challenges, such as video coding for machines. While traditional hybrid video coding can easily adapt the local rate-distortion trade-off by changing the local quantization parameter, no such approach is currently available for learning-based video compression. In this paper, we propose LRC-DHVC, a hierarchical video compression network, which allows continuous local rate control on a pixel level to vary the spatial quality distribution within individual video frames. This is achieved by concatenating a quality map to the input frame and applying a weighted MSE loss which matches the pixelwise trade-off factors in the quality map. During training, the model sees a variety of quality maps due to a constrained-random generation. Our model is the first neural video compression network, which can continuously and spatially adapt to varying quality constraints. Due to the wide quality and bit rate range, a single set of network parameters is sufficient. Compared to single rate point networks, which scale linearly with the number of rate points, the memory requirements for our network parameters remain constant. The code and model are available at link-updated-upon-acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。