arXiv:2506.03164cs.LG2025-06NeurIPS被引 29

通过优化去噪过程中的噪声路径,显著提升扩散模型生成质量。

Test-Time Scaling of Diffusion Models via Noise Trajectory Search

  • 将去噪过程建模为独立上下文老虎机问题,实现高效搜索。
  • 在文本到图像生成任务中,性能超越基线164%。
  • 无需梯度即可优化任意奖励函数,适合实际部署场景。

扩散模型的迭代与随机特性使其具备测试时扩展能力:在去噪阶段增加计算量可生成更高保真度样本。当前主要依赖增加去噪步骤,但收益迅速递减。相比之下,优化噪声轨迹(即注入噪声向量的序列)更具潜力,因具体噪声实现直接影响生成质量;然而高维搜索空间、复杂噪声-结果交互及高昂评估成本使该方法难以实施。本文首次将扩散过程建模为带终端奖励的马尔可夫决策过程(MDP),证明蒙特卡洛树搜索(MCTS)虽有效但不实用。为兼顾性能与效率,提出一种对MDP的松弛化处理,将去噪视为一系列独立上下文老虎机问题。由此引入ε-贪婪搜索算法,在极端时间步全局探索,中间阶段局部利用以完成去混叠。在EDM和Stable Diffusion上的实验显示,该方法在类别条件/文本到图像生成任务中达到当前最优表现,相较基线最高提升164%,且媲美甚至超越MCTS性能。据我们所知,这是首个针对任意非可微奖励的实用测试时噪声轨迹优化方法。

原文摘要 · Abstract (English)

The iterative and stochastic nature of diffusion models enables test-time scaling, whereby spending additional compute during denoising generates higher-fidelity samples. Increasing the number of denoising steps is the primary scaling axis, but this yields quickly diminishing returns. Instead optimizing the noise trajectory--the sequence of injected noise vectors--is promising, as the specific noise realizations critically affect sample quality; but this is challenging due to a high-dimensional search space, complex noise-outcome interactions, and costly trajectory evaluations. We address this by first casting diffusion as a Markov Decision Process (MDP) with a terminal reward, showing tree-search methods such as Monte Carlo tree search (MCTS) to be meaningful but impractical. To balance performance and efficiency, we then resort to a relaxation of MDP, where we view denoising as a sequence of independent contextual bandits. This allows us to introduce an $ε$-greedy search algorithm that globally explores at extreme timesteps and locally exploits during the intermediate steps where de-mixing occurs. Experiments on EDM and Stable Diffusion reveal state-of-the-art scores for class-conditioned/text-to-image generation, exceeding baselines by up to $164\%$ and matching/exceeding MCTS performance. To our knowledge, this is the first practical method for test-time noise trajectory optimization of arbitrary (non-differentiable) rewards.

扩散模型测试时扩展噪声优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。