改进世界模型训练，让梯度规划更快更准。
Closing the Train-Test Gap in World Models for Gradient-Based Planning
- 训练时合成数据，缩小预测与规划的差距。
- 测试时性能媲美传统方法，速度提升10倍。
- 适合需要快速决策的机器人任务场景。
将世界模型与模型预测控制（MPC）结合，可在大规模专家轨迹数据上离线训练，并在推理时泛化到多种规划任务。相比依赖慢速搜索或精确求解优化问题的传统MPC，基于梯度的规划计算效率更高。然而，其性能仍落后于其他方法。本文提出改进的世界模型训练方法，以解决训练与测试目标不一致的问题：训练时预测下一状态，但测试时需估计动作序列。为此，我们设计了训练时的数据合成技术，显著提升现有世界模型的梯度规划能力。在多种物体操作与导航任务中，该方法在10%的时间预算内，性能优于或匹配经典的无梯度交叉熵方法（CEM）。
原文摘要 · Abstract (English)
World models paired with model predictive control (MPC) can be trained offline on large-scale datasets of expert trajectories and enable generalization to a wide range of planning tasks at inference time. Compared to traditional MPC procedures, which rely on slow search algorithms or on iteratively solving optimization problems exactly, gradient-based planning offers a computationally efficient alternative. However, the performance of gradient-based planning has thus far lagged behind that of other approaches. In this paper, we propose improved methods for training world models that enable efficient gradient-based planning. We begin with the observation that although a world model is trained on a next-state prediction objective, it is used at test-time to instead estimate a sequence of actions. The goal of our work is to close this train-test gap. To that end, we propose train-time data synthesis techniques that enable significantly improved gradient-based planning with existing world models. At test time, our approach outperforms or matches the classical gradient-free cross-entropy method (CEM) across a variety of object manipulation and navigation tasks in 10% of the time budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。