通过分阶段训练提升生成模型效率，先对齐语义表示再优化生成细节。
Efficient Generative Model Training via Embedded Representation Warmup
- 分两阶段训练：先用预训练编码器对齐早期特征，再专注高保真生成。
- 350轮内达FID=1.41，速度比单阶段方法快11.5倍。
- 适合追求高效训练的生成模型研究者，尤其擅长扩散模型改进。
生成模型面临核心挑战：需同时学习高层语义概念（生成什么）与底层合成细节（如何生成）。传统端到端训练将两者纠缠，导致优化复杂低效。本文提出嵌入表示预热（ERW），一种分两阶段的训练框架。第一阶段通过将扩散模型的早期层与强大预训练编码器对齐，建立稳健的语义基础，提供强表征先验；第二阶段则以对齐损失进行完整生成训练，聚焦于高保真合成。分析表明，该方法有效性源于使模型早期层功能专用于表征学习。实验证明，本框架在350轮内达到FID=1.41，相比REPA等单阶段方法提速11.5倍。代码已开源。
原文摘要 · Abstract (English)
Generative models face a fundamental challenge: they must simultaneously learn high-level semantic concepts (what to generate) and low-level synthesis details (how to generate it). Conventional end-to-end training entangles these distinct, and often conflicting objectives, leading to a complex and inefficient optimization process. We argue that explicitly decoupling these tasks is key to unlocking more effective and efficient generative modeling. To this end, we propose Embedded Representation Warmup (ERW), a principled two-phase training framework. The first phase is dedicated to building a robust semantic foundation by aligning the early layers of a diffusion model with a powerful pretrained encoder. This provides a strong representational prior, allowing the second phase -- generative full training with alignment loss to refine the representation -- to focus its resources on high-fidelity synthesis. Our analysis confirms that this efficacy stems from functionally specializing the model's early layers for representation. Empirically, our framework achieves a 11.5$\times$ speedup in 350 epochs to reach FID=1.41 compared to single-phase methods like REPA. Code is available at https://github.com/LINs-lab/ERW.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。