让扩散模型视频超分实现低延迟流式处理，首帧仅需0.328秒。
Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion
- 基于自回归时序引导，仅用历史帧实现因果推理
- 单帧处理速度达0.328秒，比之前快130倍以上
- 适合直播、实时视频增强等低延迟场景
基于扩散模型的视频超分辨率（VSR）方法虽能提供优异的视觉质量，但因依赖未来帧和高成本的多步去噪过程，难以用于低延迟场景。我们提出Stream-DiffVSR，一种严格基于过去帧的因果扩散框架，实现高效在线VSR。该方法采用四步蒸馏去噪器加速推理，引入自回归时序引导（ARTG）模块，在潜在空间去噪过程中注入运动对齐提示，并设计轻量级时序感知解码器与时序处理器模块（TPM），提升细节与时序一致性。相比分块流式推理，其逐帧因果设计避免序列级等待，显著降低首帧时间与端到端延迟。在RTX 4090上，720p帧处理仅需0.328秒，持续优于现有扩散基基线。相较在线最新方法TMP，感知质量提升（LPIPS +0.095）；相比MGLD-VSR，单帧运行时间减少超130倍。此外，首帧延迟从超过4600秒降至0.328秒，极大提升扩散模型在低延迟流媒体部署中的实用性。
原文摘要 · Abstract (English)
Diffusion-based video super-resolution (VSR) methods deliver strong perceptual quality but are often unsuitable for latency-sensitive scenarios due to reliance on future frames and expensive multi-step denoising. We propose Stream-DiffVSR, a causally conditioned diffusion framework for efficient online VSR. Operating strictly on past frames, Stream-DiffVSR integrates a four-step distilled denoiser for fast inference, an Auto-regressive Temporal Guidance (ARTG) module that injects motion-aligned cues during latent denoising, and a lightweight temporal-aware decoder with a Temporal Processor Module (TPM) to enhance detail and temporal coherence. Unlike chunk-wise streaming inference, our strictly frame-by-frame causal design avoids sequence-level waiting, substantially reducing time-to-first-frame and end-to-end latency. Stream-DiffVSR processes 720p frames in 0.328 seconds on an RTX 4090 and consistently outperforms prior diffusion-based baselines. Compared with the online state-of-the-art TMP, it improves perceptual quality (LPIPS +0.095). Compared with prior diffusion-based VSR methods such as MGLD-VSR, it reduces per-frame runtime by over 130x. Moreover, Stream-DiffVSR substantially lowers time-to-first-frame for diffusion-based VSR, reducing initial delay from over 4600 seconds to 0.328 seconds, making diffusion-based VSR markedly more practical for low-latency online and streaming deployment. Project page: https://jamichss.github.io/stream-diffvsr-project-page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。