提出并行扩散推理方法，让强化学习模型高效生成未来画面
Horizon Imagination: Efficient On-Policy Rollout in Diffusion World Models
- 并行去噪多个未来帧，替代传统串行推理
- 仅用一半去噪步数即达同等控制性能
- 适合需要快速决策的实时强化学习场景
我们研究基于扩散模型的世界模型在强化学习中的应用，这类模型虽具备高生成质量，但在控制任务中面临严重效率瓶颈。现有方法或需重型推理模型，或依赖高度串行的想象过程，计算开销巨大。本文提出Horizon Imagination（HI），一种针对离散随机策略的在线策略想象机制，可并行去噪多个未来观测。该方法引入稳定化机制与新型采样调度，将去噪预算与有效去噪时域解耦，并支持子帧级预算。在Atari 100K与Craftium上的实验表明，本方法在仅使用一半去噪步数的子帧预算下，仍保持良好控制性能，且在多种调度下生成质量更优。代码已开源。
原文摘要 · Abstract (English)
We study diffusion-based world models for reinforcement learning, which offer high generative fidelity but face critical efficiency challenges in control. Current methods either require heavyweight models at inference or rely on highly sequential imagination, both of which impose prohibitive computational costs. We propose Horizon Imagination (HI), an on-policy imagination process for discrete stochastic policies that denoises multiple future observations in parallel. HI incorporates a stabilization mechanism and a novel sampling schedule that decouples the denoising budget from the effective horizon over which denoising is applied while also supporting sub-frame budgets. Experiments on Atari 100K and Craftium show that our approach maintains control performance with a sub-frame budget of half the denoising steps and achieves superior generation quality under varied schedules. Code is available at https://github.com/leor-c/horizon-imagination.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。