提出高效实时神经视频编码，显著提升速度且压缩率更高。
Towards Practical Real-Time Neural Video Compression
- 用隐式时序建模替代复杂运动模块，减少操作开销
- 1080p视频平均编码/解码达125.2/112.8帧每秒，比特率降低21%
- 支持跨设备整数化编码，适合实际部署场景
我们提出一种实用的实时神经视频编码器（NVC），旨在实现高压缩比、低延迟和广泛适用性。实际编码速度受计算成本与非计算操作成本（如内存读写、函数调用次数）双重影响。现有高效NVC主要关注降低计算成本，但本文识别出操作成本是制约速度提升的主要瓶颈。基于此，我们设计了一系列以效率为导向的改进：采用隐式时序建模消除复杂的显式运动模块，使用单一低分辨率潜在表示而非渐进下采样。这些创新显著提升了编码速度，同时保持压缩质量。此外，引入模型整数化实现跨设备一致编码，并采用基于模块库的码率控制方案增强实用性。实验表明，所提出的DCVC-RT在1080p视频上平均编码/解码速度达125.2/112.8帧每秒，相比H.266/VTM平均节省21%比特率。代码已公开于https://github.com/microsoft/DCVC。
原文摘要 · Abstract (English)
We introduce a practical real-time neural video codec (NVC) designed to deliver high compression ratio, low latency and broad versatility. In practice, the coding speed of NVCs depends on 1) computational costs, and 2) non-computational operational costs, such as memory I/O and the number of function calls. While most efficient NVCs prioritize reducing computational cost, we identify operational cost as the primary bottleneck to achieving higher coding speed. Leveraging this insight, we introduce a set of efficiency-driven design improvements focused on minimizing operational costs. Specifically, we employ implicit temporal modeling to eliminate complex explicit motion modules, and use single low-resolution latent representations rather than progressive downsampling. These innovations significantly accelerate NVC without sacrificing compression quality. Additionally, we implement model integerization for consistent cross-device coding and a module-bank-based rate control scheme to improve practical adaptability. Experiments show our proposed DCVC-RT achieves an impressive average encoding/decoding speed at 125.2/112.8 fps (frames per second) for 1080p video, while saving an average of 21% in bitrate compared to H.266/VTM. The code is available at https://github.com/microsoft/DCVC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。