用扩散模型修复严重压缩视频,提升画质且不乱添内容。
DiffCVE: Diffusion-based Compressed Video Enhancement

- 引入编码先验引导去噪,融合残差和运动矢量信息。
- 通过码率提示词与LoRA微调,让模型感知压缩程度。
- 适合处理高压缩比视频,对画质损伤修复效果显著。
严重压缩视频的感知质量增强因复杂的伪影模式和大量信息丢失而极具挑战。尽管扩散模型在视觉修复中表现出强大生成能力,但直接应用于压缩视频常忽略压缩退化特性,可能引入结构不一致的幻觉。为此,本文提出基于扩散的压缩视频增强方法DiffCVE。设计编码先验增强的双条件分支(CPDC),联合建模压缩视频与编码先验条件,其中残差和运动矢量为去噪过程提供互补的结构与运动指导。为使扩散过程感知压缩严重度,引入压缩退化语义提示机制(CDSP),结合QP条件文本提示与LoRA微调。此外,在VAE解码器中引入编码先验引导加权融合模块(CPWF),以QP预测权重融合编码器与先验编码器特征。大量实验表明,该方法在严重压缩场景下显著提升感知质量。项目主页含增强视频演示:https://wqmaker.github.io/projects/DiffCVE/
原文摘要 · Abstract (English)
Perceptual quality enhancement of severely compressed videos remains challenging due to complex artifact patterns and substantial information loss. Recent diffusion models have demonstrated strong generative capability for visual restoration, but directly applying them to compressed video often ignores compression degradation characteristics and may introduce structure-inconsistent hallucinations. To address this issue, this paper presents a diffusion-based compressed video enhancement method, named DiffCVE. Coding Prior-enhanced Dual Conditioning (CPDC) branches are designed to jointly model compressed video and coding prior conditions, where coding priors including residuals and motion vectors provide complementary structural and motion guidance during the diffusion denoising process. To make the diffusion process aware of compression severity, a Compression Degradation Semantic Prompting (CDSP) mechanism is introduced to leverage QP-conditioned textual prompts together with LoRA fine-tuning. In addition, a Coding Prior-guided Weighted Fusion (CPWF) module is incorporated into the VAE decoder to fuse VAE encoder and coding prior encoder features with QP-predicted weights. Extensive experiments demonstrate the effectiveness of the proposed method in improving perceptual quality, especially under severe compression settings. The project page with enhanced video demonstrations is available at https://wqmaker.github.io/projects/DiffCVE/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。