用分层预测编码实现高效视频压缩,实时运行且内存占用低。
High-Efficiency Neural Video Compression via Hierarchical Predictive Learning
- 分层变分自编码器生成多尺度特征,逐级融合空间与时间参考。
- 在1080p下实现实时编码,内存占用比传统方法减少70%以上。
- 适合网络传输,支持丢包场景下的渐进解码,适合流媒体应用。
本文提出增强版深度分层视频压缩模型DHVC 2.0。该单模型神经视频编解码器可在宽范围比特率下运行,不仅压缩性能优于现有方法,且复杂度极低,可在标准GPU上实现实时处理并显著降低内存占用。其核心为分层预测编码:每帧通过分层变分自编码器统一转换为多尺度表示;在特定尺度上,特征残差由同一帧的低尺度空间特征生成,并利用结合同尺度时间参考与当前帧低尺度空间参考的参数预测模型进行条件熵编码。处理从最低到最高尺度逐级完成,彻底摒弃了传统编解码器中耗时的运动估计与补偿。该分层结构支持并行计算,加速编码与解码,同时支持传输友好的渐进解码,对网络视频应用中的包丢失具有显著适应性。源代码将公开。
原文摘要 · Abstract (English)
The enhanced Deep Hierarchical Video Compression-DHVC 2.0-has been introduced. This single-model neural video codec operates across a broad range of bitrates, delivering not only superior compression performance to representative methods but also impressive complexity efficiency, enabling real-time processing with a significantly smaller memory footprint on standard GPUs. These remarkable advancements stem from the use of hierarchical predictive coding. Each video frame is uniformly transformed into multiscale representations through hierarchical variational autoencoders. For a specific scale's feature representation of a frame, its corresponding latent residual variables are generated by referencing lower-scale spatial features from the same frame and then conditionally entropy-encoded using a probabilistic model whose parameters are predicted using same-scale temporal reference from previous frames and lower-scale spatial reference of the current frame. This feature-space processing operates from the lowest to the highest scale of each frame, completely eliminating the need for the complexity-intensive motion estimation and compensation techniques that have been standard in video codecs for decades. The hierarchical approach facilitates parallel processing, accelerating both encoding and decoding, and supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss. Source codes will be made available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。