通过拉直生成路径提升一阶生成模型的收敛与质量。
Overcoming the Curvature Bottleneck in MeanFlow
- 用预训练模型的直线耦合修正速度场，简化均值速度估计。
- 在相同训练预算下，FID从30.9降至8.6，优于2-修正流++33.4%。
- 适合追求高效高质一阶生成模型的研究者和开发者。
MeanFlow 通过直接学习均值速度场实现一步生成建模，避免了昂贵的数值积分。然而我们发现，现有模型生成轨迹高度弯曲，导致损失曲面噪声大，严重阻碍收敛与模型质量。基于一个基本几何原理：沿直线路径估计均值速度要简单得多，我们提出 Rectified MeanFlow，一种自蒸馏方法，通过预训练模型的修正耦合，将速度场拉直并学习其均值速度。为进一步增强线性，引入基于距离的截断启发式，剔除高曲率残差对。该方法平滑了优化景观，显著提升生成性能。在相同训练预算下，基准 MeanFlow 模型的 FID 从 30.9 降低至 8.6，并在 FID 上比近期 2-修正流++ 提升 33.4%，同时运行速度快 26 倍。结果表明，一阶流生成困难部分源于弯曲轨迹带来的崎岖优化景观。代码已开源。
原文摘要 · Abstract (English)
MeanFlow offers a promising framework for one-step generative modeling by directly learning a mean-velocity field, bypassing expensive numerical integration. However, we find that the highly curved generative trajectories of existing models induce a noisy loss landscape, severely bottlenecking convergence and model quality. We leverage a fundamental geometric principle to overcome this: mean-velocity estimation is drastically simpler along straight paths. Building on this insight, we propose Rectified MeanFlow, a self-distillation approach that learns the mean-velocity field over a straightened velocity field, induced by rectified couplings from a pretrained model. To further promote linearity, we introduce a distance-based truncation heuristic that prunes residual high-curvature pairs. By smoothing the optimization landscape, our method achieves strong one-step generation performance. We improve the FID of baseline MeanFlow models from 30.9 to 8.6 under same training budget, and outperform the recent 2-rectified flow++ by 33.4% in FID while running 26x faster. Our work suggests that the difficulty of one-step flow generation stems partially from the rugged optimization landscapes induced by curved trajectories. Code is available at https://github.com/Xinxi-Zhang/Re-MeanFlow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。