arXiv:2501.09732cs.CV2025-01被引 252

通过优化采样噪声,让扩散模型在推理时用更多计算提升生成质量。

Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps

  • 设计搜索框架,用验证器反馈找更优噪声
  • 推理时增加计算量显著提升图像生成质量
  • 适配不同场景,可灵活组合组件

生成模型在多个领域产生重大影响,主要得益于训练阶段可通过增加数据、计算资源和模型规模实现可扩展性,这一现象称为缩放定律。近期研究开始探索大语言模型在推理阶段的缩放行为,揭示了推理时额外计算能进一步提升性能。与大语言模型不同,扩散模型天然可通过调整去噪步骤数来调节推理时的计算量,但性能增益通常在几十步后趋于平缓。本文研究扩散模型在增加去噪步骤之外的推理时缩放行为,探索如何通过增加计算进一步提升生成性能。具体而言,我们考虑一个寻找更好噪声的搜索问题,沿两个维度构建设计空间:用于提供反馈的验证器,以及用于寻找更优噪声候选的算法。在类别条件和文本条件图像生成基准上进行大量实验,结果表明,增加推理时计算量能显著提升扩散模型生成样本的质量;由于图像的复杂性,框架中各组件的组合可针对不同应用场景进行选择。

原文摘要 · Abstract (English)

Generative models have made significant impacts across various domains, largely due to their ability to scale during training by increasing data, computational resources, and model size, a phenomenon characterized by the scaling laws. Recent research has begun to explore inference-time scaling behavior in Large Language Models (LLMs), revealing how performance can further improve with additional computation during inference. Unlike LLMs, diffusion models inherently possess the flexibility to adjust inference-time computation via the number of denoising steps, although the performance gains typically flatten after a few dozen. In this work, we explore the inference-time scaling behavior of diffusion models beyond increasing denoising steps and investigate how the generation performance can further improve with increased computation. Specifically, we consider a search problem aimed at identifying better noises for the diffusion sampling process. We structure the design space along two axes: the verifiers used to provide feedback, and the algorithms used to find better noise candidates. Through extensive experiments on class-conditioned and text-conditioned image generation benchmarks, our findings reveal that increasing inference-time compute leads to substantial improvements in the quality of samples generated by diffusion models, and with the complicated nature of images, combinations of the components in the framework can be specifically chosen to conform with different application scenario.

扩散模型推理优化图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。