提出新方法提升神经网络视频编码中双向参考帧的利用效率
Neural B-frame Video Compression with Bi-directional Reference Harmonization
- 通过运动收敛与上下文融合优化双向参考帧
- 在HEVC数据集上超越现有神经编码，接近传统编码性能
- 适合关注视频压缩效率与神经网络编码的开发者
近年来神经视频压缩（NVC）取得显著进展，但神经B帧压缩（NBVC）相比P帧仍研究不足。NBVC可采用双向参考帧以提升压缩效果，但其分层编码在大帧距层级可能导致时间预测连续性受阻，使两个参考帧贡献失衡。为此，本文提出双向参考协调视频压缩（BRHVC），引入双向运动收敛（BMC）与双向上下文融合（BCF）。BMC在运动压缩中聚合多光流，实现更大尺度的精准运动补偿；BCF则根据运动补偿精度显式建模参考上下文权重。由此提升运动与上下文效率，有效协调双向参考。实验表明，BRHVC优于当前最优神经压缩方法，甚至在随机访问配置下超越传统编码器VTM-RA，测试基于HEVC数据集。代码已开源于https://github.com/kwai/NVC。
原文摘要 · Abstract (English)
Neural video compression (NVC) has made significant progress in recent years, while neural B-frame video compression (NBVC) remains underexplored compared to P-frame compression. NBVC can adopt bi-directional reference frames for better compression performance. However, NBVC's hierarchical coding may complicate continuous temporal prediction, especially at some hierarchical levels with a large frame span, which could cause the contribution of the two reference frames to be unbalanced. To optimize reference information utilization, we propose a novel NBVC method, termed Bi-directional Reference Harmonization Video Compression (BRHVC), with the proposed Bi-directional Motion Converge (BMC) and Bi-directional Contextual Fusion (BCF). BMC converges multiple optical flows in motion compression, leading to more accurate motion compensation on a larger scale. Then BCF explicitly models the weights of reference contexts under the guidance of motion compensation accuracy. With more efficient motions and contexts, BRHVC can effectively harmonize bi-directional references. Experimental results indicate that our BRHVC outperforms previous state-of-the-art NVC methods, even surpassing the traditional coding, VTM-RA (under random access configuration), on the HEVC datasets. The source code is released at https://github.com/kwai/NVC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。