通过6000次实验，发现扩散规划中反直觉的设计反而更有效。
What Makes a Good Diffusion Planner for Decision Making?
- 系统测试6000个模型，找出扩散规划关键组件
- 无条件采样加选择比引导采样效果更好
- Transformer比U-Net更适合做去噪网络
扩散模型在决策问题中展现出巨大潜力，尤其在生成行为计划（即扩散规划）方面。尽管已有大量研究证明其出色性能，但优秀扩散规划器的核心机制仍不清晰，设计选择也极为不一致。本文在离线强化学习（offline RL）设置下，通过系统的实证实验，揭示了扩散规划的关键组件。我们训练并评估了超过6000个扩散模型，识别出引导采样、网络结构、动作生成和规划策略等关键因素。结果表明，一些与以往实践相反的设计反而表现更优，例如无条件采样结合选择优于引导采样，Transformer作为去噪网络的表现优于U-Net。基于这些发现，我们提出一个简单而强大的扩散规划基线，在标准离线RL基准上取得了当前最优结果。
原文摘要 · Abstract (English)
Diffusion models have recently shown significant potential in solving decision-making problems, particularly in generating behavior plans -- also known as diffusion planning. While numerous studies have demonstrated the impressive performance of diffusion planning, the mechanisms behind the key components of a good diffusion planner remain unclear and the design choices are highly inconsistent in existing studies. In this work, we address this issue through systematic empirical experiments on diffusion planning in an offline reinforcement learning (RL) setting, providing practical insights into the essential components of diffusion planning. We trained and evaluated over 6,000 diffusion models, identifying the critical components such as guided sampling, network architecture, action generation and planning strategy. We revealed that some design choices opposite to the common practice in previous work in diffusion planning actually lead to better performance, e.g., unconditional sampling with selection can be better than guided sampling and Transformer outperforms U-Net as denoising network. Based on these insights, we suggest a simple yet strong diffusion planning baseline that achieves state-of-the-art results on standard offline RL benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。