arXiv:2603.18501cs.CVcs.AI2026-03被引 1

用稀疏传输提升视频压缩效率,显著改善超低码率下的画质与连贯性。

Efficient Video Diffusion with Sparse Information Transmission for Video Compression

  • 通过稀疏编码减少信息冗余,降低码率
  • 单步扩散模型结合帧类型嵌入,提升时序一致性
  • 适用于对画质和流畅性要求高的低码率视频应用

视频压缩旨在以最低码率实现最佳重建质量。除了传统失真指标外,感知质量与时序一致性同样关键。但在超低码率下,传统端到端压缩模型常产生模糊、感知质量差的图像。现有生成式压缩方法多独立处理帧,存在时序不一致和效率不足的问题。为此,我们提出高效视频扩散模型稀疏信息传输(Diff-SIT),包含稀疏时序编码模块(STEM)和单步视频扩散帧类型嵌入器(ODFTE)。STEM将原始帧序列稀疏编码为信息密集的中间序列,大幅节省码率;随后ODFTE整体处理该中间序列,利用时序相关性。过程中提出的帧类型嵌入器(FTE)引导扩散模型根据不同帧类型自适应重建,优化整体质量。在多个数据集上的实验表明,Diff-SIT在感知质量与时序一致性上达到新基准,尤其在超低码率场景表现突出。代码已开源:https://github.com/MingdeZhou/Diff-SIT。

原文摘要 · Abstract (English)

Video compression aims to maximize reconstruction quality with minimal bitrates. Beyond standard distortion metrics, perceptual quality and temporal consistency are also critical. However, at ultra-low bitrates, traditional end-to-end compression models tend to produce blurry images of poor perceptual quality. Besides, existing generative compression methods often treat video frames independently and show limitations in time coherence and efficiency. To address these challenges, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT), which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The STEM sparsely encodes the original frame sequence into an information-rich intermediate sequence, achieving significant bitrate savings. Subsequently, the ODFTE processes this intermediate sequence as a whole, which exploits the temporal correlation. During this process, our proposed Frame Type Embedder (FTE) guides the diffusion model to perform adaptive reconstruction according to different frame types to optimize the overall quality. Extensive experiments on multiple datasets demonstrate that Diff-SIT establishes a new state-of-the-art in perceptual quality and temporal consistency, particularly in the challenging ultra-low-bitrate regime. Code is released at https://github.com/MingdeZhou/Diff-SIT.

视频压缩扩散模型稀疏编码时序一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。