arXiv:2608.01397cs.ROcs.CV2026-08被引 1

提出自引导世界建模框架,让机器人在几何感知的策略空间中精准预测动作后果。

SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space

论文配图:SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
图 1 · 摘自论文原文
  • 在策略生成空间直接建模动作相关的动态变化,无需额外隐空间
  • 在LIBERO和LIBERO-Plus上分别达到98.5%和73%的成功率,优于基线
  • 适合需要高精度动作预测与环境几何理解的机器人任务

世界动作模型(WAM)将动作生成与未来状态预测结合。其有效性取决于未来动态是否在既与动作生成对齐、又具备足够几何感知能力的空间中建模。现有WAM通常只满足部分条件,依赖感知复杂的观察空间目标或非联合结构化的辅助隐空间。本文提出SG-WAM,一种自引导框架,直接在策略生成的表示空间中学习几何感知的动作条件动态。该方法引入可学习的动态标记,并设计自引导世界预测器,根据干预动作预测其未来的潜在状态。预测目标由同一策略主干的指数移动平均副本生成,提供稳定监督。几何监督进一步结构化策略图像标记表示,为动态标记提供空间上下文,构建出兼具动作相关性与几何感知性的未来对齐空间。潜在未来预测、几何锚定与流匹配动作生成在统一框架中端到端联合优化。基于0.9B参数量模型,未使用大规模具身预训练,SG-WAM在LIBERO上实现98.5%平均成功率,在LIBERO-Plus上达73%,并在分布内与分布外真实世界评估中均超越强基线。

原文摘要 · Abstract (English)

World Action Models (WAMs) couple action generation with prediction of future states. Their effectiveness depends on whether future dynamics are modeled in a space that is both aligned with action generation and sufficiently geometry-aware to capture where and how actions change the scene. Existing WAMs typically satisfy only part of this requirement, relying on either perceptually heavy observation-space targets or auxiliary latent spaces that are not jointly structured for action relevance and geometry. We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space. SG-WAM introduces learnable dynamics tokens and a Self-Guided World Predictor that forecasts their future latent states conditioned on intervening robot actions. Prediction targets are generated by an exponential moving average copy of the same policy backbone, providing stable supervision within the representation family used by the action expert. Geometric supervision further structures the policy image-token representations, providing spatially grounded context for the dynamics tokens and yielding a future-alignment space that is both action-relevant and geometry-aware. Latent future prediction, geometric grounding, and flow-matching action generation are jointly optimized end-to-end in a unified framework. Built on a 0.9B model without large-scale embodied pretraining, SG-WAM achieves 98.5% average success on LIBERO and 73% on LIBERO-Plus, while outperforming strong baselines in both in-distribution and out-of-distribution real-world evaluations.

世界建模机器人几何感知动作预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。