arXiv:2601.20308cs.CVcs.GR2026-01被引 2

用一步扩散模型提升真实场景视频时空超分辨率效果

Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusion

  • 一步式扩散框架,先线性初始化结构再分治优化时序与纹理
  • 在复杂真实退化下实现领先性能,帧率与画质同步提升
  • 适合需要高鲁棒性的视频增强应用,如监控、旧片修复

扩散模型在视频超分辨率(VSR)中表现卓越,能生成精细细节。然而其在时空视频超分辨率(STVSR)中的潜力尚未充分探索——STVSR不仅需恢复高分辨率视觉内容,还需提升帧率并保持时序连贯性。现有方法多基于简单退化假设,难以应对真实场景中复杂的未知退化。为此,本文提出OSDEnhancer,首个实现一步扩散式稳健STVSR的框架。该方法从线性初始化开始,建立必要的时空结构并适配一步重建;采用分治策略,引入时序一致性(TC)与纹理增强(TE)LoRA,分别专注帧间动态建模与细粒度纹理恢复,推理时协同优化。双向VAE解码器使用可变形循环块,利用原始VAE的多尺度结构,通过联合多尺度可变形聚合与帧间特征传播,提升从隐空间到像素的重建质量。实验表明,该方法在真实场景下达到当前最优性能,具有更强泛化能力。代码已开源:https://github.com/W-Shuoyan/OSDEnhancer。

原文摘要 · Abstract (English)

Diffusion models have demonstrated exceptional success in video super-resolution (VSR), exhibiting powerful capabilities for generating fine-grained details. However, their potential for space-time video super-resolution (STVSR), which necessitates not only recovering realistic high-resolution visual content but also improving the frame rate with coherent temporal dynamics, remains largely underexplored. Moreover, existing STVSR methods predominantly address spatiotemporal upsampling under simple degradation assumptions, thus failing in real-world scenarios with complex unknown degradations. To address these challenges, we propose OSDEnhancer, the first framework that achieves robust STVSR in one-step diffusion. OSDEnhancer begins with a linear initialization to establish essential spatiotemporal structures and adapt the model for one-step reconstruction. It then applies a divide-and-conquer strategy, introducing the temporal coherence (TC) and texture enrichment (TE) LoRAs that progressively specialize in inter-frame dynamics modeling and fine-grained texture recovery, respectively, while collaborating during inference for enhanced overall performance. A bidirectional VAE decoder employs deformable recurrent blocks to leverage the multi-scale structure of the vanilla VAE, enhancing latent-to-pixel reconstruction through joint multi-scale deformable aggregation and inter-frame feature propagation. Experimental results demonstrate that the proposed method attains state-of-the-art performance with superior generalization in real-world scenarios. The code is available at https://github.com/W-Shuoyan/OSDEnhancer.

视频超分辨率扩散模型时序建模真实场景

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。