用高压缩率VAE实现单步视频超分，速度提升百倍
Asymmetric VAE for One-Step Video Super-Resolution Acceleration
- 设计高压缩率f16 VAE，结合像素洗牌与通道复制实现高效上采样
- 训练速度比多步模型快111.9倍，比现有单步模型快3.92倍
- 适合需要极低延迟视频超分的实时应用，如直播与安防
扩散模型在真实场景视频超分辨率任务中表现出显著优势，近年基于扩散的视频超分辨率模型已将采样步骤压缩至一步，但仍存在推理效率优化空间。本文提出FastVSR，通过采用高压缩率变分自编码器（空间压缩比16，记为f16）大幅降低计算开销。我们设计了f16 VAE结构，并引入稳定训练框架。利用像素洗牌与通道复制实现额外上采样。此外，提出下界引导训练策略，以更简单的训练目标作为VAE性能的下界，提升训练稳定性与收敛性。实验表明，FastVSR相比多步模型提速111.9倍，相较现有单步模型提速3.92倍。代码与模型将开源于https://github.com/JianzeLi-114/FastVSR。
原文摘要 · Abstract (English)
Diffusion models have significant advantages in the field of real-world video super-resolution and have demonstrated strong performance in past research. In recent diffusion-based video super-resolution (VSR) models, the number of sampling steps has been reduced to just one, yet there remains significant room for further optimization in inference efficiency. In this paper, we propose FastVSR, which achieves substantial reductions in computational cost by implementing a high compression VAE (spatial compression ratio of 16, denoted as f16). We design the structure of the f16 VAE and introduce a stable training framework. We employ pixel shuffle and channel replication to achieve additional upsampling. Furthermore, we propose a lower-bound-guided training strategy, which introduces a simpler training objective as a lower bound for the VAE's performance. It makes the training process more stable and easier to converge. Experimental results show that FastVSR achieves speedups of 111.9 times compared to multi-step models and 3.92 times compared to existing one-step models. We will release code and models at https://github.com/JianzeLi-114/FastVSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。