用自动引导提升扩散模型训练效率，实测可显著改善生成质量。
Autoguided Online Data Curation for Diffusion Model Training
- 结合自动引导与在线数据选择，实现高效训练
- 相同时间内生成图像质量更高、多样性更好
- 适合追求高质量生成的模型训练场景
生成模型的计算成本重新点燃了对高效数据清洗的期待。本文研究了近期发展的自动引导与在线数据选择方法，能否提升生成扩散模型训练的时间与样本效率。我们构建了一个统一代码库，集成联合示例选择（JEST）与自动引导，支持快速消融实验与基准测试。在受控的二维合成数据生成任务及(3×64×64)-D图像生成任务上评估不同数据清洗组合，比较条件为相同运行时间与相同样本数，明确考虑选择过程的开销。实验结果表明，自动引导在所有任务中均一致提升样本质量与多样性。早期应用JEST（仅在训练初期进行选择）在数据效率上可媲美或小幅超越自动引导本身。然而，其带来的额外时间开销与复杂度使得在多数情况下，自动引导或均匀随机采样更优。这些发现表明，虽然针对性在线选择可在早期训练中带来效率增益，但提升样本质量的核心驱动力仍是自动引导。本文讨论了局限性与适用范围，并指明数据选择在何种情况下可能有益。
原文摘要 · Abstract (English)
The costs of generative model compute rekindled promises and hopes for efficient data curation. In this work, we investigate whether recently developed autoguidance and online data selection methods can improve the time and sample efficiency of training generative diffusion models. We integrate joint example selection (JEST) and autoguidance into a unified code base for fast ablation and benchmarking. We evaluate combinations of data curation on a controlled 2-D synthetic data generation task as well as (3x64x64)-D image generation. Our comparisons are made at equal wall-clock time and equal number of samples, explicitly accounting for the overhead of selection. Across experiments, autoguidance consistently improves sample quality and diversity. Early AJEST (applying selection only at the beginning of training) can match or modestly exceed autoguidance alone in data efficiency on both tasks. However, its time overhead and added complexity make autoguidance or uniform random data selection preferable in most situations. These findings suggest that while targeted online selection can yield efficiency gains in early training, robust sample quality improvements are primarily driven by autoguidance. We discuss limitations and scope, and outline when data selection may be beneficial.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。