arXiv:2506.23618cs.CV2025-06ICCV被引 3

用32倍压缩+分步条件生成,10秒内完成1080p视频超分。

TurboVSR: Fantastic Video Upscalers and Where to Find Them

  • 通过32×32×8高压缩自编码器减少令牌数,降低计算量。
  • 采用分步条件策略,仅需7秒即可处理2秒1080p视频,速度超100倍。
  • 支持4K图像超分,生成细节惊人,适合高效超分辨率应用。

基于扩散的生成模型在视频超分辨率(VSR)任务中表现出色,显著提升了细节生成能力,但面临严重的计算效率问题。例如,现有方法处理2秒1080p视频需数十分钟。本文提出TurboVSR,一种超高效的扩散模型视频超分辨率方法。核心设计包含三方面:(1) 使用压缩比达32×32×8的自编码器,大幅减少令牌数量;(2) 针对高度压缩潜在表示带来的训练挑战,引入分解式条件机制:先超分初始帧,再以高分辨率初始帧和低分辨率后续帧为条件,逐帧超分;(3) 将预训练扩散模型转化为捷径模型,减少采样步骤,加速推理。结果表明,TurboVSR性能媲美当前最优方法,且推理速度提升100倍以上,仅需7秒即可处理2秒1080p视频。该方法还可将图像视为单帧视频进行超分,4K(3648×2048)图像超分任务中展现出惊人的细节恢复能力。

原文摘要 · Abstract (English)

Diffusion-based generative models have demonstrated exceptional promise in the video super-resolution (VSR) task, achieving a substantial advancement in detail generation relative to prior methods. However, these approaches face significant computational efficiency challenges. For instance, current techniques may require tens of minutes to super-resolve a mere 2-second, 1080p video. In this paper, we present TurboVSR, an ultra-efficient diffusion-based video super-resolution model. Our core design comprises three key aspects: (1) We employ an autoencoder with a high compression ratio of 32$\times$32$\times$8 to reduce the number of tokens. (2) Highly compressed latents pose substantial challenges for training. We introduce factorized conditioning to mitigate the learning complexity: we first learn to super-resolve the initial frame; subsequently, we condition the super-resolution of the remaining frames on the high-resolution initial frame and the low-resolution subsequent frames. (3) We convert the pre-trained diffusion model to a shortcut model to enable fewer sampling steps, further accelerating inference. As a result, TurboVSR performs on par with state-of-the-art VSR methods, while being 100+ times faster, taking only 7 seconds to process a 2-second long 1080p video. TurboVSR also supports image resolution by considering image as a one-frame video. Our efficient design makes SR beyond 1080p possible, results on 4K (3648$\times$2048) image SR show surprising fine details.

视频超分扩散模型高效推理4K超分

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。