提出可复用的统一编码器,让文生图模型一步生成更逼真多样图像。
One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models
- 用共享编码器跨时间步复用特征,实现并行采样加速
- 在1步采样下生成图像多样性与真实感超越现有方法
- 适合追求高速生成且注重图像质量的应用场景
文生图扩散模型虽进展显著,但推理速度与图像质量存在权衡。现有蒸馏模型虽能减少采样步骤,却常因多样性与质量不足受限,尤其在单步生成中表现不佳。分析发现,UNet编码器存在冗余计算,而解码器更能捕捉丰富语义信息。为此,本文提出首个时间无关的统一编码器TiUE,使学生模型的编码器特征可在多个解码时间步间共享。通过单次前向传播,实现并行采样,显著降低推理时间复杂度。同时引入KL散度正则项,提升噪声预测精度,增强生成图像的感知真实感与多样性。实验表明,TiUE优于LCM、SD-Turbo与SwiftBrushv2,在仅1步采样下仍生成更高质量、更多样化的图像,兼具高效性与保真度。
原文摘要 · Abstract (English)
Text-to-Image (T2I) diffusion models have made remarkable advancements in generative modeling; however, they face a trade-off between inference speed and image quality, posing challenges for efficient deployment. Existing distilled T2I models can generate high-fidelity images with fewer sampling steps, but often struggle with diversity and quality, especially in one-step models. From our analysis, we observe redundant computations in the UNet encoders. Our findings suggest that, for T2I diffusion models, decoders are more adept at capturing richer and more explicit semantic information, while encoders can be effectively shared across decoders from diverse time steps. Based on these observations, we introduce the first Time-independent Unified Encoder TiUE for the student model UNet architecture, which is a loop-free image generation approach for distilling T2I diffusion models. Using a one-pass scheme, TiUE shares encoder features across multiple decoder time steps, enabling parallel sampling and significantly reducing inference time complexity. In addition, we incorporate a KL divergence term to regularize noise prediction, which enhances the perceptual realism and diversity of the generated images. Experimental results demonstrate that TiUE outperforms state-of-the-art methods, including LCM, SD-Turbo, and SwiftBrushv2, producing more diverse and realistic results while maintaining the computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。