用随机丢块提升扩散模型生成质量，不需重新训练。
Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Models
- 通过随机丢弃网络块构建子模型，动态修正低质预测。
- 在图文和文视频生成任务中，显著优于传统引导方法。
- 无需训练即可增强现有模型，适合快速部署应用。
Classifier-free Guidance（CFG）是现代扩散模型中广泛使用的技术，用于提升样本质量和提示遵循性。然而，通过对具有闭式解的高斯混合建模进行实证分析，我们发现CFG产生的结果次优，与真实情况存在差距。模型对这些次优预测的过度依赖常导致语义不连贯和低质量输出。为此，我们首次实证表明，可通过模型自身的子网络有效优化这些次优预测。基于此，我们提出S²-Guidance，一种在前向过程中引入随机块丢弃以构建随机子网络的新方法，从而引导模型避开潜在低质量预测，迈向高质量输出。在文本到图像和文本到视频生成任务上的大量定性和定量实验表明，S²-Guidance性能优异，持续超越CFG及其他先进引导策略。代码将公开。
原文摘要 · Abstract (English)
Classifier-free Guidance (CFG) is a widely used technique in modern diffusion models for enhancing sample quality and prompt adherence. However, through an empirical analysis on Gaussian mixture modeling with a closed-form solution, we observe a discrepancy between the suboptimal results produced by CFG and the ground truth. The model's excessive reliance on these suboptimal predictions often leads to semantic incoherence and low-quality outputs. To address this issue, we first empirically demonstrate that the model's suboptimal predictions can be effectively refined using sub-networks of the model itself. Building on this insight, we propose S$^2$-Guidance, a novel method that leverages stochastic block-dropping during the forward process to construct stochastic sub-networks, effectively guiding the model away from potential low-quality predictions and toward high-quality outputs. Extensive qualitative and quantitative experiments on text-to-image and text-to-video generation tasks demonstrate that S$^2$-Guidance delivers superior performance, consistently surpassing CFG and other advanced guidance strategies. Our code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。