提出SGG方法,提升扩散模型在无训练引导下的泛化能力。
Improving Diffusion Generalization with Weak-to-Strong Segmented Guidance
- 基于弱到强原则设计混合引导机制SGG,结合CFG与AG优势。
- 在SD3和SD3.5上推理性能超越现有无训练引导方法。
- 可迁移至训练目标,提升无引导扩散模型的泛化性,适用于条件与无条件任务。
扩散模型通过迭代精炼生成合成图像,但其模拟无关的目标与迭代过程间的不匹配常导致采样轨迹中累积梯度误差,影响结果质量与泛化能力。分类器无关引导(CFG)和AutoGuidance(AG)等引导技术通过主信号与劣质信号间的外推缓解此问题。尽管已有实证成功,但主流引导方法的有效操作范围仍不明确,导致预设条件下方法选择模糊。本文首次通过合成对比实验,从弱到强原则出发,揭示CFG与AG的有效作用区间。基于此,提出一种名为SGG的混合实例化方法,融合两者优势。进一步证明,弱到强原则结合SGG可迁移至训练目标,提升无引导扩散模型的泛化能力。全面实验验证:推理阶段,SD3与SD3.5上SGG优于现有无训练引导变体;训练阶段,基于变换器架构的实验显示,在条件与无条件设置下均实现有效迁移与性能提升。代码已开源。
原文摘要 · Abstract (English)
Diffusion models generate synthetic images through an iterative refinement process. However, the misalignment between the simulation-free objective and the iterative process often causes accumulated gradient error along the sampling trajectory, which leads to unsatisfactory results and a failure to generalize. Guidance techniques like Classifier Free Guidance (CFG) and AutoGuidance (AG) alleviate this by extrapolating between the main and inferior signal for stronger generalization. Despite empirical success, the effective operational regimes of prevalent guidance methods are still under-explored, leading to ambiguity when selecting the appropriate guidance method given a precondition. In this work, we first conduct synthetic comparisons to isolate and demonstrate the effective regime of guidance methods represented by CFG and AG from the perspective of weak-to-strong principle. Based on this, we propose a hybrid instantiation called SGG under the principle, taking the benefits of both. Furthermore, we demonstrate that the W2S principle along with SGG can be migrated into the training objective, improving the generalization ability of unguided diffusion models. We validate our approach with comprehensive experiments. At inference time, evaluations on SD3 and SD3.5 confirm that SGG outperforms existing training-free guidance variants. Training-time experiments on transformer architectures demonstrate the effective migration and performance gains in both conditional and unconditional settings. Code is available at https://github.com/851695e35/SGG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。