arXiv:2507.18192cs.CV2025-07ICCV被引 1

用融合文本嵌入的方法,让模型生成更快更省算力。

TeEFusion: Blending Text Embeddings to Distill Classifier-Free Guidance

  • 将条件与无条件文本嵌入线性融合,直接注入引导强度。
  • 学生模型推理速度提升6倍,图像质量接近教师模型。
  • 适合追求高效生成的开发者和部署场景。

最近的文本到图像生成进展主要得益于复杂的采样策略和无分类器引导(CFG),但其依赖两次前向传播,尤其在结合复杂采样算法时,导致推理成本极高。为此,我们提出 TeEFusion(文本嵌入融合)方法,一种新颖高效的蒸馏技术,将引导幅度直接融入文本嵌入,并蒸馏教师模型复杂的采样策略。通过简单的线性操作融合条件与无条件文本嵌入,TeEFusion 在不增加额外参数的情况下重建所需引导,使学生模型能够学习教师模型使用复杂采样策略生成的输出。在 SD3 等先进模型上的大量实验表明,该方法使学生模型能以更简单高效的采样策略逼近教师性能。结果表明,学生模型推理速度最高可达教师模型的6倍,同时保持与教师模型相当的图像质量。代码已公开于 https://github.com/AIDC-AI/TeEFusion。

原文摘要 · Abstract (English)

Recent advances in text-to-image synthesis largely benefit from sophisticated sampling strategies and classifier-free guidance (CFG) to ensure high-quality generation. However, CFG's reliance on two forward passes, especially when combined with intricate sampling algorithms, results in prohibitively high inference costs. To address this, we introduce TeEFusion (Text Embeddings Fusion), a novel and efficient distillation method that directly incorporates the guidance magnitude into the text embeddings and distills the teacher model's complex sampling strategy. By simply fusing conditional and unconditional text embeddings using linear operations, TeEFusion reconstructs the desired guidance without adding extra parameters, simultaneously enabling the student model to learn from the teacher's output produced via its sophisticated sampling approach. Extensive experiments on state-of-the-art models such as SD3 demonstrate that our method allows the student to closely mimic the teacher's performance with a far simpler and more efficient sampling strategy. Consequently, the student model achieves inference speeds up to 6$\times$ faster than the teacher model, while maintaining image quality at levels comparable to those obtained through the teacher's complex sampling approach. The code is publicly available at https://github.com/AIDC-AI/TeEFusion.

文本生成扩散模型加速推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。