针对视频超分中扩散模型的冗余问题,提出高效单步模型OASIS。
Towards Redundancy Reduction in Diffusion Models for Efficient Video Super-Resolution
- 通过注意力专精路由,让不同注意力头专注不同模式,减少冗余计算。
- 在真实世界数据集上达到顶尖性能,推理速度比基线快6.2倍。
- 适合追求高速高质视频超分的工业应用与实时系统开发。
扩散模型在视频超分辨率(VSR)任务中表现优异,但直接将其应用于低质量视频会引入冗余,因为原始视频已包含大量内容信息。这种冗余导致计算开销和学习负担增加,模型需进行多余操作并过滤无关信息。为此,本文提出OASIS——一种用于真实世界视频超分辨率的单步扩散模型,其具备注意力专精路由机制,可根据注意力头的内在行为分配至不同模式。该机制有效缓解冗余,同时保留预训练知识,使模型更适配VSR任务并提升性能。此外,提出一种简单有效的渐进式训练策略:先在时间一致退化条件下训练,再过渡到不一致设置,从而更好学习复杂退化下的特征。大量实验表明,OASIS在合成与真实世界数据集上均达当前最优性能,且推理速度相比单步扩散基线(如SeedVR2)提升6.2倍。代码将公开于https://github.com/jp-guo/OASIS。
原文摘要 · Abstract (English)
Diffusion models have recently shown promising results for video super-resolution (VSR). However, directly adapting generative diffusion models to VSR can result in redundancy, since low-quality videos already preserve substantial content information. Such redundancy leads to increased computational overhead and learning burden, as the model performs superfluous operations and must learn to filter out irrelevant information. To address this problem, we propose OASIS, an efficient $\textbf{o}$ne-step diffusion model with $\textbf{a}$ttention $\textbf{s}$pecialization for real-world v$\textbf{i}$deo $\textbf{s}$uper-resolution. OASIS incorporates an attention specialization routing that assigns attention heads to different patterns according to their intrinsic behaviors. This routing mitigates redundancy while effectively preserving pretrained knowledge, allowing diffusion models to better adapt to VSR and achieve stronger performance. Moreover, we propose a simple yet effective progressive training strategy, which starts with temporally consistent degradations and then shifts to inconsistent settings. This strategy facilitates learning under complex degradations. Extensive experiments demonstrate that OASIS achieves state-of-the-art performance on both synthetic and real-world datasets. OASIS also provides superior inference speed, offering a $\textbf{6.2$\times$}$ speedup over one-step diffusion baselines such as SeedVR2. The code will be available at \href{https://github.com/jp-guo/OASIS}{https://github.com/jp-guo/OASIS}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。