解析一致性模型训练不稳根源，提出稳定优化新方法
Stabilizing Consistency Training: A Flow Map Analysis and Self-Distillation
- 从流映射视角分析模型不稳定性机制
- 提出改进自蒸馏法，抑制梯度爆炸提升收敛性
- 适用于图像生成与策略学习，无需预训练模型
一致性模型因其快速生成能力，已达到与扩散模型和流模型相当的性能。然而，从零训练时存在固有的不稳定性和可复现性差的问题，尽管已有研究尝试解释并稳定训练过程，但现有解释仍零散，理论关联不清晰。本文从流映射角度对一致性模型进行理论分析，阐明了训练稳定性与收敛行为如何导致退化解。基于此，我们重新审视自蒸馏作为缓解特定次优收敛的实用方案，并重构其形式以避免过大的梯度范数，实现稳定优化。实验表明，该方法不仅适用于图像生成，还可扩展至基于扩散的策略学习，且无需依赖预训练扩散模型初始化，展现出更广泛的应用潜力。
原文摘要 · Abstract (English)
Consistency models have been proposed for fast generative modeling, achieving results competitive with diffusion and flow models. However, these methods exhibit inherent instability and limited reproducibility when training from scratch, motivating subsequent work to explain and stabilize these issues. While these efforts have provided valuable insights, the explanations remain fragmented, and the theoretical relationships remain unclear. In this work, we provide a theoretical examination of consistency models by analyzing them from a flow map-based perspective. This joint analysis clarifies how training stability and convergence behavior can give rise to degenerate solutions. Building on these insights, we revisit self-distillation as a practical remedy for certain forms of suboptimal convergence and reformulate it to avoid excessive gradient norms for stable optimization. We demonstrate that our strategy extends beyond image generation to diffusion-based policy learning, without reliance on pretrained diffusion models for initialization, illustrating its broader applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。