提出单步扩散模型DOVE，实现真实视频超分的高效推理。
DOVE: Efficient One-Step Diffusion Model for Real-World Video Super-Resolution
- 通过潜空间-像素联合训练策略微调预训练视频扩散模型。
- 在HQ-VSR数据集上微调后，性能媲美多步方法，推理快28倍。
- 适合追求实时视频超分的工业应用或移动端部署。
扩散模型在真实世界视频超分辨率（VSR）中表现优异，但需数十次采样步骤，导致推理极慢。单步采样加速技术是潜在解决方案，但在VSR中仍具挑战，因视频数据训练开销大且保真度要求高。为此，我们提出DOVE——一种高效的单步扩散模型用于真实世界视频超分辨率。DOVE通过微调预训练视频扩散模型CogVideoX获得。为有效训练，我们提出潜空间-像素联合训练策略，采用两阶段渐进式适应机制。同时，设计了针对VSR优化的高质量数据处理流程，构建了名为HQ-VSR的数据集，进一步提升模型恢复能力。大量实验表明，DOVE性能可与多步扩散方法相当甚至更优，推理速度相比MGLD-VSR等方法最高提升28倍。代码已公开于https://github.com/zhengchen1999/DOVE。
原文摘要 · Abstract (English)
Diffusion models have demonstrated promising performance in real-world video super-resolution (VSR). However, the dozens of sampling steps they require, make inference extremely slow. Sampling acceleration techniques, particularly single-step, provide a potential solution. Nonetheless, achieving one step in VSR remains challenging, due to the high training overhead on video data and stringent fidelity demands. To tackle the above issues, we propose DOVE, an efficient one-step diffusion model for real-world VSR. DOVE is obtained by fine-tuning a pretrained video diffusion model (i.e., CogVideoX). To effectively train DOVE, we introduce the latent-pixel training strategy. The strategy employs a two-stage scheme to gradually adapt the model to the video super-resolution task. Meanwhile, we design a video processing pipeline to construct a high-quality dataset tailored for VSR, termed HQ-VSR. Fine-tuning on this dataset further enhances the restoration capability of DOVE. Extensive experiments show that DOVE exhibits comparable or superior performance to multi-step diffusion-based VSR methods. It also offers outstanding inference efficiency, achieving up to a 28$\times$ speed-up over existing methods such as MGLD-VSR. Code is available at: https://github.com/zhengchen1999/DOVE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。