提出混合时序传播机制，用小缓冲实现高效视频编码
HyTIP: Hybrid Temporal Information Propagation for Masked Conditional Residual Video Coding
- 结合解码帧与隐层特征双重传播，平衡性能与延迟
- 在同等性能下缓冲区仅需传统方法一小部分，实测小3倍以上
- 适合低延迟场景，尤其对内存受限设备友好
多数基于帧的自学习视频编码可视为沿时间维度传播参考信息的循环神经网络（RNN）。本文从RNN视角重新审视现有方法的局限性。输出递归方法虽直观，但对解码帧施加双重约束，导致率失真性能不佳；隐藏状态递归方法灵活性更高，却需大缓存。为此，我们提出HyTIP，一种融合两种机制的视频编码框架。其混合缓存策略同时使用显式解码帧和少量隐式潜在特征，在保持优异编码性能的同时大幅降低缓存需求。实验表明，相比单一使用任一机制的方法，HyTIP表现更优；在性能接近当前最优方法的前提下，缓存显著更小，并在PSNR-RGB和MS-SSIM-RGB指标上超越VTM 17.0（低延迟B模式）。
原文摘要 · Abstract (English)
Most frame-based learned video codecs can be interpreted as recurrent neural networks (RNNs) propagating reference information along the temporal dimension. This work revisits the limitations of the current approaches from an RNN perspective. The output-recurrence methods, which propagate decoded frames, are intuitive but impose dual constraints on the output decoded frames, leading to suboptimal rate-distortion performance. In contrast, the hidden-to-hidden connection approaches, which propagate latent features within the RNN, offer greater flexibility but require large buffer sizes. To address these issues, we propose HyTIP, a learned video coding framework that combines both mechanisms. Our hybrid buffering strategy uses explicit decoded frames and a small number of implicit latent features to achieve competitive coding performance. Experimental results show that our HyTIP outperforms the sole use of either output-recurrence or hidden-to-hidden approaches. Furthermore, it achieves comparable performance to state-of-the-art methods but with a much smaller buffer size, and outperforms VTM 17.0 (Low-delay B) in terms of PSNR-RGB and MS-SSIM-RGB. The source code of HyTIP is available at https://github.com/NYCU-MAPL/HyTIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。