单卡训练高质图像生成，仅用1个批次实现高效扩散蒸馏。
High Quality Diffusion Distillation on a Single GPU with Relative and Absolute Position Matching
- 通过相对与绝对位置匹配模拟教师模型采样轨迹。
- 单卡、单批次训练下，4步即可达到最佳方法1步的生成质量。
- 适合资源有限的研究者快速复现高质量扩散模型。
我们提出相对与绝对位置匹配（RAPM），一种可在单张GPU上高效训练的扩散蒸馏方法，实现高质量图像生成。近期扩散蒸馏研究如分阶段一致性模型（PCM）和改进分布匹配蒸馏（DMD2）在高分辨率文生图任务中表现优异，但通常需要8至64张GPU及128至2048的大批量，计算与显存需求超出许多研究者的承受范围。RAPM实现了单卡、单批次（batchsize=1）训练下的有效蒸馏。该方法通过匹配教师模型的相对与绝对位置来模拟其采样轨迹，其中相对位置设计受PCM启发，并引入两个判别器分别用于相对与绝对位置匹配。在StableDiffusion V1.5与SDXL上的实验表明，使用4个步数的RAPM，在极低算力条件下生成结果的FID分数可媲美最佳方法在1个步数下的表现。
原文摘要 · Abstract (English)
We introduce relative and absolute position matching (RAPM), a diffusion distillation method resulting in high quality generation that can be trained efficiently on a single GPU. Recent diffusion distillation research has achieved excellent results for high-resolution text-to-image generation with methods such as phased consistency models (PCM) and improved distribution matching distillation (DMD2). However, these methods generally require many GPUs (e.g.~8-64) and significant batchsizes (e.g.~128-2048) during training, resulting in memory and compute requirements that are beyond the resources of some researchers. RAPM provides effective single-GPU diffusion distillation training with a batchsize of 1. The new method attempts to mimic the sampling trajectories of the teacher model by matching the relative and absolute positions. The design of relative positions is inspired by PCM. Two discriminators are introduced accordingly in RAPM, one for matching relative positions and the other for absolute positions. Experimental results on StableDiffusion (SD) V1.5 and SDXL indicate that RAPM with 4 timesteps produces comparable FID scores as the best method with 1 timestep under very limited computational resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。