用对抗训练让扩散模型一步生成视频，又快又清晰。
Diffusion Adversarial Post-Training for One-Step Video Generation
- 在预训练扩散模型后，用对抗方式微调实现单步生成。
- 1280x720、24fps的2秒视频可单步实时生成。
- 适合需要快速生成高质量视频的研究与应用。
扩散模型广泛用于图像和视频生成，但其迭代生成过程缓慢且计算开销大。尽管已有蒸馏方法在图像领域实现了单步生成，但仍存在显著质量下降问题。本文提出一种在扩散预训练后，针对真实数据进行对抗性后训练（APT）的方法，用于单步视频生成。为提升训练稳定性和生成质量，我们改进了模型架构与训练流程，并引入近似R1正则化目标。实验表明，所提出的Seaweed-APT模型可在单次前向推理中实时生成2秒、1280x720、24fps的视频；同时也能在单步内生成1024px图像，质量接近当前最先进水平。
原文摘要 · Abstract (English)
The diffusion models are widely used for image and video generation, but their iterative generation process is slow and expansive. While existing distillation approaches have demonstrated the potential for one-step generation in the image domain, they still suffer from significant quality degradation. In this work, we propose Adversarial Post-Training (APT) against real data following diffusion pre-training for one-step video generation. To improve the training stability and quality, we introduce several improvements to the model architecture and training procedures, along with an approximated R1 regularization objective. Empirically, our experiments show that our adversarial post-trained model, Seaweed-APT, can generate 2-second, 1280x720, 24fps videos in real time using a single forward evaluation step. Additionally, our model is capable of generating 1024px images in a single step, achieving quality comparable to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。