arXiv:2505.19958cs.CV2025-05被引 14

用单步扩散空间实现逼真且连贯的视频超分辨率

UltraVSR: Achieving Ultra-Realistic Video Super-Resolution with Efficient One-Step Diffusion Space

  • 通过退化感知重建调度,将多步去噪简化为单步重构
  • 在单次采样下达成当前最佳效果,保持时序一致性
  • 适合追求高效高质视频增强的开发者与研究者

扩散模型在生成逼真图像细节方面展现出巨大潜力。然而,由于其固有的随机性和缺乏时序建模能力,将其应用于视频超分辨率(VSR)仍具挑战。以往方法尝试通过引入运动信息和时序层缓解该问题,但低分辨率视频中不可靠的运动估计以及深层时序层带来的高计算成本,限制了其在长序列上的应用。本文提出UltraVSR,一种通过高效单步扩散空间实现超逼真且时序一致的VSR新框架。核心组件为退化感知重建调度(DRS),从低分辨率输入估计退化因子,将迭代去噪过程转化为从低分辨率到高分辨率的单步重构。为保证时序一致性,提出轻量级循环时序移位(RTS)模块,包含RTS卷积单元和RTS注意力单元,通过沿时间维度部分移位特征分量,实现帧间有效传播、融合与对齐,无需显式时序层。RTS模块集成于预训练文本到图像扩散模型,并通过时空联合蒸馏(SJD)进一步提升时序一致性,同时保留真实细节。此外,引入时序异步推理(TAI)策略,在有限内存下捕捉长时序依赖。大量实验表明,UltraVSR在单次采样下实现当前最优性能,兼具定性与定量优势。代码已开源。

原文摘要 · Abstract (English)

Diffusion models have shown great potential in generating realistic image detail. However, adapting these models to video super-resolution (VSR) remains challenging due to their inherent stochasticity and lack of temporal modeling. Previous methods have attempted to mitigate this issue by incorporating motion information and temporal layers. However, unreliable motion estimation from low-resolution videos and costly multiple sampling steps with deep temporal layers limit them to short sequences. In this paper, we propose UltraVSR, a novel framework that enables ultra-realistic and temporally-coherent VSR through an efficient one-step diffusion space. A central component of UltraVSR is the Degradation-aware Reconstruction Scheduling (DRS), which estimates a degradation factor from the low-resolution input and transforms the iterative denoising process into a single-step reconstruction from low-resolution to high-resolution videos. To ensure temporal consistency, we propose a lightweight Recurrent Temporal Shift (RTS) module, including an RTS-convolution unit and an RTS-attention unit. By partially shifting feature components along the temporal dimension, it enables effective propagation, fusion, and alignment across frames without explicit temporal layers. The RTS module is integrated into a pretrained text-to-image diffusion model and is further enhanced through Spatio-temporal Joint Distillation (SJD), which improves temporally coherence while preserving realistic details. Additionally, we introduce a Temporally Asynchronous Inference (TAI) strategy to capture long-range temporal dependencies under limited memory constraints. Extensive experiments show that UltraVSR achieves state-of-the-art performance, both qualitatively and quantitatively, in a single sampling step. Code is available at https://github.com/yongliuy/UltraVSR.

视频超分扩散模型时序一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。