提出简单高效的视频超分基线,提升生成分辨率与质量。
SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution
- 设计两种退化策略,使训练数据更贴近基础模型输出
- 通过时序采样与噪声增强分析,优化模型性能
- 引入交错时序单元和稀疏注意力,降低计算开销
潜在扩散模型已成为高效视频生成的主流范式。随着用户对高分辨率输出的需求上升,仅依赖潜在空间计算已显不足。一种有前景的方法是将过程分为两阶段:语义内容生成与细节合成。前者在低分辨率下使用计算量大的基础模型,后者则通过轻量级级联视频超分辨率(VSR)模型实现高分辨率输出。本文聚焦于级联VSR模型的关键设计原则,目前研究尚不充分。首先,提出两种退化策略,生成更符合基础模型输出特性的训练对,确保VSR模型与上游生成器的一致性。其次,通过系统分析(1)时间步采样策略、(2)噪声增强对低分辨率输入的影响,揭示VSR模型行为规律,并指导架构与训练改进。最后,引入交错时序单元与稀疏局部注意力,显著降低训练与推理的计算开销。大量实验表明,该框架优于现有方法,消融实验验证了各项设计的有效性。本工作建立了一个简单而有效的级联视频超分辨率基线,为高效级联合成系统提供实用指导。
原文摘要 · Abstract (English)
Latent diffusion models have emerged as a leading paradigm for efficient video generation. However, as user expectations shift toward higher-resolution outputs, relying solely on latent computation becomes inadequate. A promising approach involves decoupling the process into two stages: semantic content generation and detail synthesis. The former employs a computationally intensive base model at lower resolutions, while the latter leverages a lightweight cascaded video super-resolution (VSR) model to achieve high-resolution output. In this work, we focus on studying key design principles for latter cascaded VSR models, which are underexplored currently. First, we propose two degradation strategies to generate training pairs that better mimic the output characteristics of the base model, ensuring alignment between the VSR model and its upstream generator. Second, we provide critical insights into VSR model behavior through systematic analysis of (1) timestep sampling strategies, (2) noise augmentation effects on low-resolution (LR) inputs. These findings directly inform our architectural and training innovations. Finally, we introduce interleaving temporal unit and sparse local attention to achieve efficient training and inference, drastically reducing computational overhead. Extensive experiments demonstrate the superiority of our framework over existing methods, with ablation studies confirming the efficacy of each design choice. Our work establishes a simple yet effective baseline for cascaded video super-resolution generation, offering practical insights to guide future advancements in efficient cascaded synthesis systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。