用一维隐变量压缩视频，显著降低码率且保持画质。
Generative Video Compression with One-Dimensional Latent Representation
- 将视频编码为一维隐向量，打破二维网格结构束缚。
- 在HEVC B类数据集上，码率降低60.4%（LPIPS）至68.8%（DISTS）。
- 适合追求高效视频压缩与低延迟传输的系统应用。
生成式视频编码（GVC）通常将视频编码为二维隐变量网格，并使用高容量生成解码器进行重建。然而，该范式仍存在两大挑战：空间上，刚性二维结构不可避免地保留帧内冗余，相邻块高度相似，导致需更高码率；时间上，二维隐变量难以紧凑且语义连贯地建模长时相关性，阻碍跨帧内容聚合。为此，本文提出基于一维（1D）隐变量表示的生成式视频压缩方法（GVC1D）。GVC1D将视频数据编码为依赖短时与长时上下文的一维隐令牌，摆脱刚性空间对应关系，使隐令牌可自适应关注语义区域，自然实现令牌压缩，有效减少空间冗余。此外，所提一维记忆机制提供语义丰富的长期上下文，同时保持低计算成本，进一步降低时间冗余。实验表明，GVC1D在压缩效率上表现卓越，在HEVC Class B数据集上，于LPIPS指标下码率降低60.4%，于DISTS指标下降低68.8%，超越现有视频压缩方法。
原文摘要 · Abstract (English)
Recent advancements in generative video codec (GVC) typically encode video into a 2D latent grid and employ high-capacity generative decoders for reconstruction. However, this paradigm still leaves two key challenges in fully exploiting spatial-temporal redundancy: Spatially, the 2D latent grid inevitably preserves intra-frame redundancy due to its rigid structure, where adjacent patches remain highly similar, thereby necessitating a higher bitrate. Temporally, the 2D latent grid is less effective for modeling long-term correlations in a compact and semantically coherent manner, as it hinders the aggregation of common contents across frames. To address these limitations, we introduce Generative Video Compression with One-Dimensional (1D) Latent Representation (GVC1D). GVC1D encodes the video data into extreme compact 1D latent tokens conditioned on both short- and long-term contexts. Without the rigid 2D spatial correspondence, these 1D latent tokens can adaptively attend to semantic regions and naturally facilitate token reduction, thereby reducing spatial redundancy. Furthermore, the proposed 1D memory provides semantically rich long-term context while maintaining low computational cost, thereby further reducing temporal redundancy. Experimental results indicate that GVC1D attains superior compression efficiency, where it achieves bitrate reductions of 60.4\% under LPIPS and 68.8\% under DISTS on the HEVC Class B dataset, surpassing the previous video compression methods.Project: https://gvc1d.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。