无需标签或额外训练,用鞍点正曲率引导生成高质量图像。
Saddle-Free Guidance: Improved On-Manifold Sampling without Labels or Additional Training
- 利用对数密度在鞍点区域的正曲率作为生成引导信号。
- 在ImageNet-512上达到最佳FID和FD-DINOv2分数。
- 兼容现有扩散与流匹配模型,适合无标签场景使用。
基于得分的生成模型需要引导以生成合理且位于流形上的样本。目前最流行的引导方法——无分类器引导(CFG)仅适用于有标签数据,并需训练额外的无条件得分模型。近期的Auto-Guidance则使用原模型的小版本进行引导。尽管这些方法有效提升生成质量,但均依赖标签数据或额外训练,难以在无标签数据或无法训练新模型时应用。本文发现:对数密度在鞍点区域的正曲率可提供强引导信号。基于此,我们提出无鞍点引导(SFG),通过保持对数密度最大正曲率估计来引导单个得分模型。SFG计算成本与CFG相当,无需额外训练,可直接用于现成的扩散与流匹配模型。实验表明,SFG在单模型无条件ImageNet-512生成中达到最优的FID与FD-DINOv2指标;结合Auto-Guidance后,其无条件生成样本在FD-DINOv2评分上达到通用领先水平。在FLUX.1-dev与Stable Diffusion v3.5上的实验显示,相比CFG,SFG提升了输出多样性,同时保持优秀提示遵循性与图像保真度。
原文摘要 · Abstract (English)
Score-based generative models require guidance in order to generate plausible, on-manifold samples. The most popular guidance method, Classifier-Free Guidance (CFG), is only applicable in settings with labeled data and requires training an additional unconditional score-based model. More recently, Auto-Guidance adopts a smaller, less capable version of the original model to guide generation. While each method effectively promotes the fidelity of generated data, each requires labeled data or the training of additional models, making it challenging to guide score-based models when (labeled) training data are not available or training new models is not feasible. We make the surprising discovery that the positive curvature of log density estimates in saddle regions provides strong guidance for score-based models. Motivated by this, we develop saddle-free guidance (SFG) which maintains estimates of maximal positive curvature of the log density to guide individual score-based models. SFG has the same computational cost of classifier-free guidance, does not require additional training, and works with off-the-shelf diffusion and flow matching models. Our experiments indicate that SFG achieves state-of-the-art FID and FD-DINOv2 metrics in single-model unconditional ImageNet-512 generation. When SFG is combined with Auto-Guidance, its unconditional samples achieve general state-of-the-art in FD-DINOv2 score. Our experiments with FLUX.1-dev and Stable Diffusion v3.5 indicate that SFG boosts the diversity of output images compared to CFG while maintaining excellent prompt adherence and image fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。