用动作预测提升世界模型，让机器人操控更高效。
Enhancing Policy Learning with World-Action Model
- 在世界模型中加入动作反向预测，让表征更懂控制
- 操控任务成功率从79.8%提升至92.8%，减少8.7倍训练步数
- 无需改架构，可直接增强现有策略学习方法
本文提出世界-动作模型（WAM），一种通过动作正则化的世界模型，联合推理未来的视觉观测与驱动状态转移的动作。不同于仅依赖图像预测的传统世界模型，WAM 在 DreamerV2 中引入反向动力学目标，从潜在状态转移中预测动作，促使学习到的表征捕捉对下游控制至关重要的动作相关结构。我们在 CALVIN 基准的八个操作任务上评估了 WAM 对策略学习的增强效果。先在世界模型潜空间上用行为克隆预训练扩散策略，再在冻结的世界模型内使用基于模型的 PPO 进行微调。无需修改策略架构或训练流程，WAM 将行为克隆平均成功率从 DreamerV2 与 DiWA 基线的 59.4% 提升至 71.2%。经过 PPO 微调后，WAM 实现 92.8% 的平均成功率，对比基线 79.8%，其中两个任务达到 100%，且训练步数减少 8.7 倍。
原文摘要 · Abstract (English)
This paper presents the World-Action Model (WAM), an action-regularized world model that jointly reasons over future visual observations and the actions that drive state transitions. Unlike conventional world models trained solely via image prediction, WAM incorporates an inverse dynamics objective into DreamerV2 that predicts actions from latent state transitions, encouraging the learned representations to capture action-relevant structure critical for downstream control. We evaluate WAM on enhancing policy learning across eight manipulation tasks from the CALVIN benchmark. We first pretrain a diffusion policy via behavioral cloning on world model latents, then refine it with model-based PPO inside the frozen world model. Without modifying the policy architecture or training procedure, WAM improves average behavioral cloning success from 59.4% to 71.2% over DreamerV2 and DiWA baselines. After PPO fine-tuning, WAM achieves 92.8% average success versus 79.8% for the baseline, with two tasks reaching 100%, using 8.7x fewer training steps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。