用图神经网络增强视频隐式表示,提升压缩效率
Video Compression with Graph-inspired Neural Representation

- 构建帧间邻域并传递信息,显式利用时间冗余
- 在UVG数据集上比NVRC快8.86%,比VVC快14.68%(BD-rate)
- 适合做神经视频压缩的科研与工程人员参考
隐式神经表示(INR)为视频压缩提供了紧凑且自适应的范式,通常通过共享网络参数和帧索引嵌入来表示视频。相比传统或基于自编码器的编解码器,这些方法以隐式方式利用视频中的时间冗余,可能导致压缩性能不佳。本文提出G-NeRV,一种受图结构启发的INR,显式增强隐式潜在空间中的时间冗余利用。受信息论中总相关性原理启发,我们在帧嵌入上构建时间邻域,并通过自适应门控机制进行消息传递,聚合相邻帧的可复用信息。受传统视频编码中参考帧缓冲区的启发,设计了记忆库机制,支持在随机帧索引采样下高效检索时间邻居。该新表示模型被集成到先进表示压缩框架中,与现有传统及神经视频编解码器对比。结果表明,G-NeRV编解码器在UVG数据集上以PSNR衡量,比最先进的INR编解码器NVRC和最新标准编解码器VVC VTM分别提升8.86%和14.68%(BD-rate)。
原文摘要 · Abstract (English)
Implicit Neural Representations (INR) provide a compact and content-adaptive paradigm for video compression, typically representing a video through shared network parameters and frame-indexed embeddings. Compared to conventional or autoencoder-based codecs, these approaches exploit temporal redundancy within videos in an implicit manner, which potentially results in sub-optimal compression performance. In this paper, we propose G-NeRV, a graph-inspired INR that explicitly improves temporal redundancy exploitation in the implicit latent space. Motivated by the total correlation principles in information theory, we construct a temporal neighborhood over frame embeddings and perform message passing to aggregate reusable information from neighboring frames through an adaptive gate controlling the injection of neighboring information. Inspired by the reference frame buffer in conventional video coding, a memory bank mechanism has been further designed to enable efficient temporal-neighbor retrieval under random frame-index sampling in INR training. This new representation model has been integrated into an advanced representation compression framework and compared with existing conventional and neural video codecs. The results show that the G-NeRV codec outperforms the state-of-the-art INR-based codec, NVRC, and the latest standard video codec, VVC VTM, by 8.86\% and 14.68\% (in BD-rate), respectively, measured by PSNR on the UVG dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。