通过上下文调制提升神经视频压缩效率,显著降低码率
Neural Video Compression with Context Modulation
- 用参考帧引导生成定向时序上下文,增强帧间相关性建模
- 在H.266/VVC基础上平均降低22.7%码率,优于现有神经编码器10.1%
- 适合追求高效率视频压缩的开发者与研究人员
高效视频编码高度依赖于利用时间冗余,通常通过基于条件编码的神经视频编解码器(NVC)提取并利用时间上下文实现。尽管最新NVC在压缩性能上取得显著进展,但其固有的时间上下文传播机制难以充分挖掘参考帧信息,制约了进一步提升。本文提出两步式上下文调制方法:首先引入流方向性,挖掘参考帧与预测帧间的关联性,生成额外的定向时间上下文;其次提出上下文补偿机制,利用定向上下文调制由传播参考特征生成的传播时间上下文。通过协同机制与解耦损失监督,有效消除无关传播信息,实现更优的上下文建模。实验表明,该编解码器在平均上比先进传统编码器H.266/VVC降低22.7%码率,相较此前最先进神经编码器DCVC-FM平均节省10.1%码率。代码已公开于https://github.com/Austin4USTC/DCMVC。
原文摘要 · Abstract (English)
Efficient video coding is highly dependent on exploiting the temporal redundancy, which is usually achieved by extracting and leveraging the temporal context in the emerging conditional coding-based neural video codec (NVC). Although the latest NVC has achieved remarkable progress in improving the compression performance, the inherent temporal context propagation mechanism lacks the ability to sufficiently leverage the reference information, limiting further improvement. In this paper, we address the limitation by modulating the temporal context with the reference frame in two steps. Specifically, we first propose the flow orientation to mine the inter-correlation between the reference frame and prediction frame for generating the additional oriented temporal context. Moreover, we introduce the context compensation to leverage the oriented context to modulate the propagated temporal context generated from the propagated reference feature. Through the synergy mechanism and decoupling loss supervision, the irrelevant propagated information can be effectively eliminated to ensure better context modeling. Experimental results demonstrate that our codec achieves on average 22.7% bitrate reduction over the advanced traditional video codec H.266/VVC, and offers an average 10.1% bitrate saving over the previous state-of-the-art NVC DCVC-FM. The code is available at https://github.com/Austin4USTC/DCMVC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。