将推理过程移到视觉语言模型隐空间,实现高效精准的机器人动作规划。
PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

- 把计划推理放到冻结的隐空间中,通过迭代修正生成精细动作方案。
- 在LIBERO基准上达到当前最佳性能,比基线提速30%以上。
- 适合需要低延迟、高精度动作规划的机器人任务场景。
现有视觉-语言-动作(VLA)模型在动作生成效率与显式规划之间存在权衡。直接从视觉语言主干特征解码动作可实现低延迟控制,而通过文本链、像素级子目标或动作搜索进行显式推理虽能提升规划质量,却带来显著延迟和计算开销。本文提出PearlVLA,将推理过程迁移至视觉语言模型(VLM)的隐空间。该框架将VLM元查询表示分为固定视觉定位分支与迭代隐计划分支。每轮精炼中,基于计划的全局查询向轻量级冻结隐世界模型获取无动作未来观测隐表示,并反馈用于指导计划优化。后续由未来引导的RefineNet执行调度残差更新,逐步将粗略语义草稿细化为细粒度隐动作计划。经过K轮迭代后,最终计划并行解码为动作块,实现低延迟执行。此外,引入因果精炼-分组过程-奖励强化学习(Causal Refinement-Grouped Process-Reward RL),利用隐计划修改诱发的长时程想象未来来优化隐空间精炼过程。在LIBERO基准上的实证评估表明,PearlVLA在现有方法中表现最优。
原文摘要 · Abstract (English)
Current Vision-Language-Action (VLA) models face a trade-off between efficient action generation and explicit deliberation. Directly decoding actions from vision-language backbone representations enables low-latency control, whereas explicit reasoning through textual chains, pixel-level subgoals, or action search can improve planning but incurs substantial latency and computational cost. We propose PearlVLA, a VLA framework that moves deliberation into the latent space of a vision-language model (VLM). PearlVLA separates VLM meta-query representations into a fixed visual grounding branch and an iterative latent plan branch. At each refinement round, a plan-conditioned world query probes a lightweight frozen latent world model for an action-free future observation latent, which is fed back to guide plan refinement. A future-guided RefineNet then applies scheduled residual updates to progressively refine a coarse semantic draft into a fine-grained latent action plan. The refined plan after K rounds is then decoded in parallel into an action chunk for low-latency execution. We further introduce Causal Refinement-Grouped Process-Reward RL to optimize the latent refinement process with rewards from longer-horizon imagined futures induced by latent plan edits. Empirical evaluations on the LIBERO benchmark demonstrate that PearlVLA achieves state-of-the-art performance among existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。