用优化文本嵌入替代扩散模型中的引导机制，提速同时保质。
DICE: Distilling Classifier-Free Guidance into Text Embeddings
- 通过精炼文本嵌入模拟无分类器引导方向
- 采样速度提升一倍，图像质量接近原版
- 适合追求高效生成的视觉应用开发者
文生图扩散模型能生成高质量图像，但预训练文本表示不佳常导致图像与提示词对不齐。无分类器引导（CFG）虽能有效提升对齐度，却带来显著计算开销。本文提出DICE方法，将基于CFG的模型蒸馏为无CFG版本，仅需一半计算复杂度即可保持相近生成质量。DICE通过优化文本嵌入来复现CFG的方向，避免其计算代价，实现高速、高对齐度的图像生成。进一步分析表明，DICE通过锐化文本嵌入中特定成分，在保留语义信息的同时增强细粒度细节。在Stable Diffusion v1.5多个变体、SDXL和PixArt-α上的实验验证了方法有效性。代码已公开于https://github.com/zju-pi/dice。
原文摘要 · Abstract (English)
Text-to-image diffusion models are capable of generating high-quality images, but suboptimal pre-trained text representations often result in these images failing to align closely with the given text prompts. Classifier-free guidance (CFG) is a popular and effective technique for improving text-image alignment in the generative process. However, CFG introduces significant computational overhead. In this paper, we present DIstilling CFG by sharpening text Embeddings (DICE) that replaces CFG in the sampling process with half the computational complexity while maintaining similar generation quality. DICE distills a CFG-based text-to-image diffusion model into a CFG-free version by refining text embeddings to replicate CFG-based directions. In this way, we avoid the computational drawbacks of CFG, enabling high-quality, well-aligned image generation at a fast sampling speed. Furthermore, examining the enhancement pattern, we identify the underlying mechanism of DICE that sharpens specific components of text embeddings to preserve semantic information while enhancing fine-grained details. Extensive experiments on multiple Stable Diffusion v1.5 variants, SDXL, and PixArt-$α$ demonstrate the effectiveness of our method. Code is available at https://github.com/zju-pi/dice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。