无需训练，在测试时用SMC方法让扩散模型精准对齐目标，不损失多样性。
Test-time Alignment of Diffusion Models without Reward Over-optimization
- 基于SMC和温度调节的测试时对齐方法，无需微调。
- 在单目标、多目标及黑盒优化中表现媲美甚至超越微调方法。
- 适合需要快速适配新任务又保持生成多样性的场景。
扩散模型在生成任务中表现优异，但如何在保持其通用性的同时对齐特定目标仍具挑战。现有微调方法常出现奖励过优化问题，而近似引导方法则难以有效优化目标奖励。针对上述局限，我们提出一种无需训练、基于序列蒙特卡洛（SMC）的测试时方法，用于从奖励对齐的目标分布中采样。该方法专为扩散采样设计，并引入温度调节技术，在单目标优化、多目标场景及在线黑盒优化中均达到或优于微调方法的奖励表现，同时保持生成多样性与跨奖励泛化能力。本工作为扩散模型对齐多样化下游目标提供了稳健方案，且不损害其通用性能。代码已开源：https://github.com/krafton-ai/DAS。
原文摘要 · Abstract (English)
Diffusion models excel in generative tasks, but aligning them with specific objectives while maintaining their versatility remains challenging. Existing fine-tuning methods often suffer from reward over-optimization, while approximate guidance approaches fail to optimize target rewards effectively. Addressing these limitations, we propose a training-free, test-time method based on Sequential Monte Carlo (SMC) to sample from the reward-aligned target distribution. Our approach, tailored for diffusion sampling and incorporating tempering techniques, achieves comparable or superior target rewards to fine-tuning methods while preserving diversity and cross-reward generalization. We demonstrate its effectiveness in single-reward optimization, multi-objective scenarios, and online black-box optimization. This work offers a robust solution for aligning diffusion models with diverse downstream objectives without compromising their general capabilities. Code is available at https://github.com/krafton-ai/DAS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。