用噪声重构替代生成引导，提速省内存。
A Noise is Worth Diffusion Guidance
- 将初始噪声优化为无引导可用的高质量噪声
- 仅需5万图文对训练，即可实现无引导高质生成
- 适合追求高效推理与低资源部署的研究者
扩散模型在生成高质量图像方面表现优异，但当前模型依赖分类器无关引导（CFG）等方法才能稳定输出。本文观察到，通过扩散反演获得的噪声可重建高质量图像而无需引导。由此聚焦去噪流程的初始噪声，提出将标准高斯噪声映射为“无引导噪声”。研究发现，微小的低频低幅成分显著提升去噪效果，从而摆脱对引导的依赖，提高推理速度并降低内存占用。为此，我们提出 exttt{NoiseRefine}，仅通过一次初始噪声精修，即可在不修改原扩散流程的前提下实现无引导高质量生成。该噪声精修模型采用高效的噪声空间学习，仅用5万文本-图像对即快速收敛，性能强劲。我们在多种指标上验证了其有效性，并分析了优化后噪声如何消除对引导的需求。
原文摘要 · Abstract (English)
Diffusion models excel in generating high-quality images. However, current diffusion models struggle to produce reliable images without guidance methods, such as classifier-free guidance (CFG). Are guidance methods truly necessary? Observing that noise obtained via diffusion inversion can reconstruct high-quality images without guidance, we focus on the initial noise of the denoising pipeline. By mapping Gaussian noise to `guidance-free noise', we uncover that small low-magnitude low-frequency components significantly enhance the denoising process, removing the need for guidance and thus improving both inference throughput and memory. Expanding on this, we propose \ours, a novel method that replaces guidance methods with a single refinement of the initial noise. This refined noise enables high-quality image generation without guidance, within the same diffusion pipeline. Our noise-refining model leverages efficient noise-space learning, achieving rapid convergence and strong performance with just 50K text-image pairs. We validate its effectiveness across diverse metrics and analyze how refined noise can eliminate the need for guidance. See our project page: https://cvlab-kaist.github.io/NoiseRefine/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。