通过分阶段采样提升流匹配模型训练效率与质量
Curriculum Sampling: A Two-Phase Curriculum for Efficient Training of Flow Matching
- 先用中部集中采样快速学结构，再切换到均匀采样精修边界
- 在CIFAR-10上将FID从3.85降至3.22，提前至10万步达峰值
- 适合追求高效高质生成模型的开发者和研究者
时间步采样分布p(t)是流匹配模型的核心设计之一，当前普遍采用静态中段偏倚分布（如Logit-Normal）。我们发现这种选择带来速度与质量的权衡：中段偏倚采样虽加速早期收敛，但最终保真度低于均匀采样。通过对各时间步训练损失的分析，我们识别出呈U形的难度分布，边界区域存在持续误差，表明端点采样不足导致细节未被充分优化。基于此，我们提出「课程采样」（Curriculum Sampling），采用两阶段策略：初期使用中段偏倚采样快速学习整体结构，后期切换为均匀采样以精细优化边界。在CIFAR-10上，该方法将最佳FID从3.85（均匀采样）提升至3.22，并在10万训练步时达到峰值性能，而非原需15万步。结果表明，时间步采样应被视为动态演进的课程，而非固定超参数。
原文摘要 · Abstract (English)
Timestep sampling $p(t)$ is a central design choice in Flow Matching models, yet common practice increasingly favors static middle-biased distributions (e.g., Logit-Normal). We show that this choice induces a speed--quality trade-off: middle-biased sampling accelerates early convergence but yields worse asymptotic fidelity than Uniform sampling. By analyzing per-timestep training losses, we identify a U-shaped difficulty profile with persistent errors near the boundary regimes, implying that under-sampling the endpoints leaves fine details unresolved. Guided by this insight, we propose \textbf{Curriculum Sampling}, a two-phase schedule that begins with middle-biased sampling for rapid structure learning and then switches to Uniform sampling for boundary refinement. On CIFAR-10, Curriculum Sampling improves the best FID from $3.85$ (Uniform) to $3.22$ while reaching peak performance at $100$k rather than $150$k training steps. Our results highlight that timestep sampling should be treated as an evolving curriculum rather than a fixed hyperparameter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。