arXiv:2603.00458cs.CV2026-03被引 4

将扩散模型压缩为轻量级网络,实现快速高清视频修复。

Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution

  • 用3D注意力蒸馏大模型,再结合1D时序卷积提升效率。
  • 参数减少95%,推理速度提升8倍,画质仍保持领先。
  • 双头对抗机制分离细节与连贯性优化,适合实时视频修复场景。

尽管许多扩散模型在真实世界视频超分辨率(Real-VSR)中生成了丰富逼真的细节,但其多步采样导致推理缓慢。类似SeedVR2、DOVE和DLoRAL的一步网络虽通过单步生成缓解该问题,但仍存在参数量达数十亿、延迟超过秒级的瓶颈。近期提出的对抗式扩散压缩(ADC)通过剪枝与蒸馏,将模型压缩为紧凑的AdcSR网络,但直接应用于Real-VSR时因缺乏时序感知能力及标准对抗学习限制,难以平衡空间细节与时间一致性。为此,我们提出改进的ADC方法:将具备3D时空注意力的大规模扩散Transformer(DiT)教师模型DOVE,蒸馏至一个剪枝后的2D稳定扩散(SD)骨干网络,并引入轻量级1D时序卷积,显著提升效率。此外,设计双头对抗蒸馏方案,使像素与特征域判别器分别专注细节与一致性,实现二者协同优化。实验表明,所获压缩模型在参数量上降低95%,相较教师模型DOVE实现8倍加速,同时保持优异的视频质量与效率。

原文摘要 · Abstract (English)

While many diffusion models have achieved impressive results in real-world video super-resolution (Real-VSR) by generating rich and realistic details, their reliance on multi-step sampling leads to slow inference. One-step networks like SeedVR2, DOVE, and DLoRAL alleviate this through condensing generation into one single step, yet they remain heavy, with billions of parameters and multi-second latency. Recent adversarial diffusion compression (ADC) offers a promising path via pruning and distilling these models into a compact AdcSR network, but directly applying it to Real-VSR fails to balance spatial details and temporal consistency due to its lack of temporal awareness and the limitations of standard adversarial learning. To address these challenges, we propose an improved ADC method for Real-VSR. Our approach distills a large diffusion Transformer (DiT) teacher DOVE equipped with 3D spatio-temporal attentions, into a pruned 2D Stable Diffusion (SD)-based AdcSR backbone, augmented with lightweight 1D temporal convolutions, achieving significantly higher efficiency. In addition, we introduce a dual-head adversarial distillation scheme, in which discriminators in both pixel and feature domains explicitly disentangle the discrimination of details and consistency into two heads, enabling both objectives to be effectively optimized without sacrificing one for the other. Experiments demonstrate that the resulting compressed AdcVSR model reduces complexity by 95% in parameters and achieves an 8$\times$ acceleration over its DiT teacher DOVE, while maintaining competitive video quality and efficiency.

视频超分扩散模型模型压缩时序建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。