用视觉语言模型提升机器人决策可靠性,通过预判动作后果来筛选行动方案。
From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment
- 分离预见与判断:先用隐空间模型预演动作后果,再让VLM在语义层面评估
- 在多种机械臂任务中实现稳定策略引导,显著降低部署时失败率
- 适合关注机器人自主决策与多模态推理的研究者和开发者
生成式机器人策略虽能从示范中学习复杂多模态行为,但在部署时仍存在多种失败。策略引导通过外部验证器从生成策略提出的低层动作中选择最优方案,有效降低失败概率。理想情况下可使用视觉语言模型(VLM)作为验证器,利用其开放世界的推理能力。然而,现成的VLM难以理解低层机器人动作的后果,因其表示方式与训练数据中的文本和图像差异显著。为此,我们提出FOREWARN框架,使VLM作为开放词汇验证器在运行时实现策略引导。核心思想是将预测动作后果(预见)与评估(远见)解耦:利用隐空间世界模型,基于多样化的低层动作计划想象未来隐状态;再将VLM与这些预测的隐状态对齐,使其在自然语言本征表示下推理动作后果并有效过滤提议方案。我们在多种机器人操作任务中验证了该框架,证明其能弥合表征鸿沟,提供稳健且可泛化的策略引导。视频展示见项目主页:https://yilin-wu98.github.io/forewarn/
原文摘要 · Abstract (English)
While generative robot policies have demonstrated significant potential in learning complex, multimodal behaviors from demonstrations, they still exhibit diverse failures at deployment-time. Policy steering offers an elegant solution to reducing the chance of failure by using an external verifier to select from low-level actions proposed by an imperfect generative policy. Here, one might hope to use a Vision Language Model (VLM) as a verifier, leveraging its open-world reasoning capabilities. However, off-the-shelf VLMs struggle to understand the consequences of low-level robot actions as they are represented fundamentally differently than the text and images the VLM was trained on. In response, we propose FOREWARN, a novel framework to unlock the potential of VLMs as open-vocabulary verifiers for runtime policy steering. Our key idea is to decouple the VLM's burden of predicting action outcomes (foresight) from evaluation (forethought). For foresight, we leverage a latent world model to imagine future latent states given diverse low-level action plans. For forethought, we align the VLM with these predicted latent states to reason about the consequences of actions in its native representation--natural language--and effectively filter proposed plans. We validate our framework across diverse robotic manipulation tasks, demonstrating its ability to bridge representational gaps and provide robust, generalizable policy steering. Videos can be found on the project website: https://yilin-wu98.github.io/forewarn/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。