融合局部全局信息，用低比特实现更准的视频压缩运动补偿。
Hybrid Local-Global Context Learning for Neural Video Compression
- 大尺度用光流引导变形卷积，小尺度用光流扭曲，平衡精度与开销。
- 在标准数据集上显著提升现有神经视频编码性能，尤其在复杂场景下表现更好。
- 适合关注视频压缩效率与质量的开发者，尤其对低码率应用有帮助。
在神经视频编解码中，当前最先进方法通常采用多尺度运动补偿来处理多样化的运动模式。这些方法通过估计并压缩光流或可变形偏移量来减少帧间冗余。然而，基于光流的方法在复杂场景中常出现运动估计不准确的问题；而基于可变形卷积的方法虽更鲁棒，但运动编码的比特开销较高。本文提出一种混合上下文生成模块，以最优方式结合两者优势，在低比特成本下实现高精度补偿。具体而言，针对不同尺度特征的特点，最大尺度使用光流引导的可变形补偿，以在细节区域实现精确对齐；较小尺度则采用基于光流的扭曲策略，降低运动编码比特开销。此外，我们设计了局部-全局上下文增强模块，充分挖掘先前重建信号的局部与全局信息。实验结果表明，所提出的混合局部-全局上下文学习（HLGC）方法能在标准测试数据集上显著提升现有先进方法的性能。
原文摘要 · Abstract (English)
In neural video codecs, current state-of-the-art methods typically adopt multi-scale motion compensation to handle diverse motions. These methods estimate and compress either optical flow or deformable offsets to reduce inter-frame redundancy. However, flow-based methods often suffer from inaccurate motion estimation in complicated scenes. Deformable convolution-based methods are more robust but have a higher bit cost for motion coding. In this paper, we propose a hybrid context generation module, which combines the advantages of the above methods in an optimal way and achieves accurate compensation at a low bit cost. Specifically, considering the characteristics of features at different scales, we adopt flow-guided deformable compensation at largest-scale to produce accurate alignment in detailed regions. For smaller-scale features, we perform flow-based warping to save the bit cost for motion coding. Furthermore, we design a local-global context enhancement module to fully explore the local-global information of previous reconstructed signals. Experimental results demonstrate that our proposed Hybrid Local-Global Context learning (HLGC) method can significantly enhance the state-of-the-art methods on standard test datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。