arXiv:2606.19729cs.ROcs.AI2026-06

用扩散模型学通用不确定性规划模型,让机器人高效决策。

VOiLA: Vectorized Online Planning with Learned Diffusion Models for POMDP Agents

论文配图:VOiLA: Vectorized Online Planning with Learned Diffusion Models for POMDP Agents
图 1 · 摘自论文原文
  • 用条件扩散模型学状态转移和观测采样,生成可在线使用的模型
  • 推理速度提升近1000倍,仅用10%数据达更好性能
  • 适合想用仿真训练、落地真实机器人的研究者

在不确定性环境下进行规划是自主机器人的重要能力。部分可观测马尔可夫决策过程(POMDP)为此提供了强大框架,但其实际应用常受限于难以构建准确的POMDP模型。本文提出面向POMDP代理的向量化在线规划框架VOiLA,通过条件扩散模型学习任务无关的转移与观测采样器,并学习观测似然模型以支持基于粒子的信念更新。为实现高效在线规划,将扩散采样器压缩为轻量前馈生成器,并与专为GPU并行设计的向量化在线POMDP规划器(VOPP)集成。实验表明,该压缩策略使采样开销降低近三个数量级,使生成式POMDP模型适用于在线规划。在三个基准任务上的评估显示,VOiLA性能不低于循环软演员-评论家(Recurrent SAC),且训练数据不足其10%,对未见环境配置具有更强泛化能力。物理机器人测试中,仅使用仿真数据训练的模型在10次尝试中成功完成任务。

原文摘要 · Abstract (English)

Planning under uncertainty is an essential capability for autonomous robots. The Partially Observable Markov Decision Process (POMDP) provides a powerful framework for such a capability. Although POMDP-based planning has advanced significantly, its application to real-world problems is often limited by the difficulty of obtaining faithful POMDP models. We present Vectorized Online planning wIth Learned diffusion model for POMDP Agents (VOiLA), a framework that learns task-agnostic POMDP models for online planning under uncertainty. VOiLA learns transition and observation samplers using conditional diffusion models and learns observation-likelihood models for particle-based belief updates. To enable efficient online planning, the diffusion samplers are distilled into compact feedforward generators and integrated with Vectorized Online POMDP Planner (VOPP), an online POMDP planner designed to leverage GPU parallelization. Experimental results indicate the distillation strategy reduces sampling cost by up to nearly three orders of magnitude, making learned generative POMDP models practical for online planning. Evaluation of VOiLA on three benchmark problems indicate that VOiLA achieves equal or better performance than Recurrent Soft Actor Critic while using less than 10% training data, and generalizes much better to unseen environment configurations. Physical robot evaluation indicates VOiLA uses the models learned using only simulated data and generates a policy that successfully accomplish the task in 10 of 10 runs.

强化学习机器人扩散模型在线规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。