arXiv:2410.14040cs.LGcs.AI2024-10被引 3

用世界模型辅助生成闭环策略,提速降耗还更稳。

WARPD: World model Assisted Reactive Policy Diffusion

  • 直接生成神经网络权重,不走轨迹预测路线。
  • 长时序任务表现更好,扰动下仍稳定,错误累积少。
  • 推理耗能仅1/45,适合高频控制机器人场景。

随着开源机器人数据增多,模仿学习在操作与运动任务中愈发重要。扩散模型被广泛用于训练大型通用策略,以预测控制或轨迹,因其能建模多模态动作分布。但这种泛化能力伴随模型体积大、推理慢的问题,对需要高控制频率的机器人任务尤为不利。此外,主流的扩散策略(DP)存在性能与动作时长的权衡:减少扩散查询会导致更大轨迹块,进而累积跟踪误差。为此,我们提出WARPD(世界模型辅助的反应式策略扩散),直接生成闭环策略(神经网络权重),而非开环轨迹。通过在参数空间而非轨迹空间学习行为分布,WARPD具有两大优势:(1)在保持高任务性能的同时,实现更长的动作时长且对扰动更具鲁棒性;(2)显著降低推理成本。实证表明,WARPD在长时序和受扰环境中的表现优于DP,且在多任务上达到与DP相当的性能,但每步推理所需浮点运算量仅为后者的约1/45。

原文摘要 · Abstract (English)

With the increasing availability of open-source robotic data, imitation learning has become a promising approach for both manipulation and locomotion. Diffusion models are now widely used to train large, generalized policies that predict controls or trajectories, leveraging their ability to model multimodal action distributions. However, this generality comes at the cost of larger model sizes and slower inference, an acute limitation for robotic tasks requiring high control frequencies. Moreover, Diffusion Policy (DP), a popular trajectory-generation approach, suffers from a trade-off between performance and action horizon: fewer diffusion queries lead to larger trajectory chunks, which in turn accumulate tracking errors. To overcome these challenges, we introduce WARPD (World model Assisted Reactive Policy Diffusion), a method that generates closed-loop policies (weights for neural policies) directly, instead of open-loop trajectories. By learning behavioral distributions in parameter space rather than trajectory space, WARPD offers two major advantages: (1) extended action horizons with robustness to perturbations, while maintaining high task performance, and (2) significantly reduced inference costs. Empirically, WARPD outperforms DP in long-horizon and perturbed environments, and achieves multitask performance on par with DP while requiring only ~ 1/45th of the inference-time FLOPs per step.

机器人扩散模型策略学习高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。