让扩散模型生成更符合人类意图的图像，速度提升9.5倍
Lookahead Sample Reward Guidance for Test-Time Scaling of Diffusion Models
- 用预训练模型的采样结果直接计算奖励，无需反向传播
- 引入前瞻采样和精确求解器，实现高效高质生成
- 适合需要快速生成高质量内容的AI绘图应用
扩散模型生成效果虽强，但常与人类意图不符。本文提出一种高效的测试时扩展方法，引导采样进入高人类对齐奖励区域。现有方法中，反向滚动采样成本过高，基于Tweedie的方法存在偏差和采样问题。我们发现，任意时刻的期望未来奖励（EFR）可仅通过预训练扩散模型的边缘采样闭式计算，无需神经网络反向传播。为进一步提升效率，引入几步前瞻采样与精准求解器，引导粒子向高奖励前瞻样本靠近。该采样方案称为LiDAR。LiDAR在SDXL上达到与最新梯度引导方法相当的GenEval性能，同时实现9.5倍加速。代码已开源：https://github.com/aailab-kaist/Diffusion-LiDAR-Sampling。
原文摘要 · Abstract (English)
Diffusion models have demonstrated strong generative performance; however, generated samples often fail to fully align with human intent. This paper studies an efficient test-time scaling method for sampling from regions with higher human-aligned reward values. Existing methods for computing the expected future reward (EFR) face important limitations: backward rollout incurs prohibitively high sampling costs, while Tweedie-based approaches, including Sequential Monte Carlo and gradient guidance, suffer from bias and inherent sampling issues. We show that the EFR at any $\mathbf{x}_t$ can be computed using only marginal samples from a pre-trained diffusion model, enabling closed-form reward guidance without neural backpropagation. To further improve efficiency, we introduce a few-step lookahead sampling and an accurate solver that guides particles toward high-reward lookahead samples. We refer to this sampling scheme as LiDAR sampling. LiDAR achieves the same GenEval performance as the latest gradient guidance method for SDXL with a 9.5x speedup. We release the code at https://github.com/aailab-kaist/Diffusion-LiDAR-Sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。