提出高效时空感知的视频超分模型，显著提升真实场景下视频画质与推理速度。
DiffST: Spatiotemporal-Aware Diffusion for Real-World Space-Time Video Super-Resolution
- 采用单步采样和整视频处理，提升推理效率
- 引入跨帧信息聚合与视频级全局引导，增强时空建模能力
- 在真实视频超分任务中表现领先，速度比前代快17倍
基于扩散模型在视频超分辨率（VSR）和视频帧插值（VFI）中表现出色，但在耦合的空间-时间视频超分辨率（STVSR）任务中应用受限。现有扩散模型方法存在两个问题：（1）推理效率低；（2）时空信息利用不足。为解决这些问题，本文提出DiffST，一种面向真实世界STVSR的高效时空感知视频扩散框架。为提升效率，采用预训练扩散模型进行单步采样，并直接处理整段视频而非逐帧操作。为增强时空信息利用，设计跨帧上下文聚合（CFCA）模块以整合多关键帧信息生成中间帧，以及视频表示引导（VRG）模块提取视频级全局特征指导扩散过程。大量实验表明，DiffST在真实世界STVSR任务中取得领先性能，且推理速度相较之前的方法提升约17倍。代码已开源：https://github.com/zhengchen1999/DiffST。
原文摘要 · Abstract (English)
Diffusion-based models have shown strong performance in video super-resolution (VSR) and video frame interpolation (VFI). However, their role in the coupled space-time video super-resolution (STVSR) setting remains limited. Existing diffusion-based STVSR approaches suffer from two issues: (1) low inference efficiency and (2) insufficient utilization of spatiotemporal information. These limitations impede deployment. To address these issues, we introduce DiffST, an efficient spatiotemporal-aware video diffusion framework for real-world STVSR. To improve efficiency, we adapt a pre-trained diffusion model for one-step sampling and process the entire video directly rather than operating on individual frames. Furthermore, to enhance spatiotemporal information utilization, we introduce cross-frame context aggregation (CFCA) and video representation guidance (VRG). The CFCA module aggregates information across multiple keyframes to produce intermediate frames. The VRG module extracts video-level global features to guide the diffusion process. Extensive experiments show that DiffST obtains leading results on real-world STVSR tasks. It also maintains high inference efficiency, running about 17$\times$ faster than previous diffusion-based STVSR methods. Code is available at: https://github.com/zhengchen1999/DiffST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。