统一帧内帧间编码，实现高效实时视频压缩
Real-Time Neural Video Compression with Unified Intra and Inter Coding
- 单模型自适应处理帧内帧间编码，统一框架设计
- 相比DCVC-RT平均降低12.1%的BD-rate，稳定性更优
- 适合需要实时压缩与高质量输出的视频应用
近年来，神经视频压缩（NVC）技术迅速发展，涌现出如DCVC-RT等先进方案，其压缩效率优于H.266/VVC，并具备实时编码解码能力。然而，现有NVC方法仍存在遮挡区域和新内容处理效率低、帧间误差传播与累积等问题。为此，本文借鉴经典视频编码思想，在帧间编码帧中引入帧内编码工具，有效处理遮挡与新增内容，并自然阻断误差传播，无需手动刷新机制。我们提出一种统一帧内/帧间编码的NVC框架，所有帧由单一模型自适应处理。此外，设计了同时压缩两帧的方案，双向利用帧间冗余。实验表明，该方案相比DCVC-RT平均降低12.1%的BD-rate，比特率与质量每帧更稳定，且保持实时性能。代码与模型将公开。
原文摘要 · Abstract (English)
Neural video compression (NVC) technologies have advanced rapidly in recent years, yielding state-of-the-art schemes such as DCVC-RT that offer superior compression efficiency to H.266/VVC and real-time encoding/decoding capabilities. Nonetheless, existing NVC schemes have several limitations, including inefficiency in dealing with disocclusion and new content, interframe error propagation and accumulation, among others. To eliminate these limitations, we borrow the idea from classic video coding schemes, which allow intra coding within inter-coded frames. With the intra coding tool enabled, disocclusion and new content are properly handled, and interframe error propagation is naturally intercepted without the need for manual refresh mechanisms. We present an NVC framework with unified intra and inter coding, where every frame is processed by a single model that is trained to perform intra/inter coding adaptively. Moreover, we propose a simultaneous two-frame compression design to exploit interframe redundancy not only forwardly but also backwardly. Experimental results show that our scheme outperforms DCVC-RT by an average of 12.1% BD-rate reduction, delivers more stable bitrate and quality per frame, and retains real-time encoding/decoding performances. Code and models will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。