通过单轨迹蒸馏加速图像视频风格迁移，提升生成质量与一致性。
Single Trajectory Distillation for Accelerating Image and Video Style Transfer
- 从特定噪声状态出发，用单轨迹蒸馏训练快速一致模型。
- 在图像和视频风格迁移上，风格相似度与美学评分优于现有方法。
- 引入轨迹库与非对称对抗损失，兼顾速度与生成质量。
基于扩散模型的风格化方法通常从特定部分噪声状态开始去噪，用于图像到图像及视频到视频任务。这一多步扩散过程计算成本高，限制了实际应用。一种有前景的加速方案是通过轨迹蒸馏获得少步一致性模型。然而，现有方法仅强制学生模型与不完美教师模型在初始步骤的概率流微分方程（PF-ODE）轨迹对齐，无法保证整体轨迹一致性。为此，我们提出从特定部分噪声状态出发的单轨迹蒸馏（STD）。引入轨迹库存储教师模型的轨迹状态，降低训练时间开销。此外，采用非对称对抗损失提升生成图像的风格与质量。大量实验表明，本方法在图像与视频风格迁移任务中，于风格相似度与美学评估上均超越现有加速模型。代码与结果将公开于项目主页：https://single-trajectory-distillation.github.io。
原文摘要 · Abstract (English)
Diffusion-based stylization methods typically denoise from a specific partial noise state for image-to-image and video-to-video tasks. This multi-step diffusion process is computationally expensive and hinders real-world application. A promising solution to speed up the process is to obtain few-step consistency models through trajectory distillation. However, current consistency models only force the initial-step alignment between the probability flow ODE (PF-ODE) trajectories of the student and the imperfect teacher models. This training strategy can not ensure the consistency of whole trajectories. To address this issue, we propose single trajectory distillation (STD) starting from a specific partial noise state. We introduce a trajectory bank to store the teacher model's trajectory states, mitigating the time cost during training. Besides, we use an asymmetric adversarial loss to enhance the style and quality of the generated images. Extensive experiments on image and video stylization demonstrate that our method surpasses existing acceleration models in terms of style similarity and aesthetic evaluations. Our code and results will be available on the project page: https://single-trajectory-distillation.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。