arXiv:2602.10983cs.RO2026-02被引 11

用大模型分解视觉任务,让机器人更懂复杂操作。

Scaling World Model for Hierarchical Manipulation Policies

  • 高层用世界模型生成带图像的目标,低层用VLA执行
  • 新场景下成功率从14%提升至69%
  • 适合做通用机器人操作的科研与工程人员

视觉-语言-动作(VLA)模型在通用机器人操作中前景广阔,但在分布外(OOD)场景下仍易失效,尤其在真实数据有限时。为突破泛化瓶颈,我们提出一种分层视觉-语言-动作框架 our{},利用大规模预训练世界模型实现鲁棒且可泛化的视觉子目标任务分解(VISTA)。该框架由高层世界模型作为规划器,低层VLA作为执行器:高层先根据目标图像将任务分解为子任务序列,低层则依据文本和视觉提示生成动作序列。相比纯文本目标,合成的目标图像提供了具象的视觉与物理信息,使低层策略能有效泛化到未见过的物体和新场景。我们在大量分布外场景中验证了视觉目标合成与分层VLA策略的有效性,相同结构的VLA在新场景下的性能从14%提升至69%。结果表明,该方法显著优于以往基线,尤其在分布外场景中表现突出。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models are promising for generalist robot manipulation but remain brittle in out-of-distribution (OOD) settings, especially with limited real-robot data. To resolve the generalization bottleneck, we introduce a hierarchical Vision-Language-Action framework \our{} that leverages the generalization of large-scale pre-trained world model for robust and generalizable VIsual Subgoal TAsk decomposition VISTA. Our hierarchical framework \our{} consists of a world model as the high-level planner and a VLA as the low-level executor. The high-level world model first divides manipulation tasks into subtask sequences with goal images, and the low-level policy follows the textual and visual guidance to generate action sequences. Compared to raw textual goal specification, these synthesized goal images provide visually and physically grounded details for low-level policies, making it feasible to generalize across unseen objects and novel scenarios. We validate both visual goal synthesis and our hierarchical VLA policies in massive out-of-distribution scenarios, and the performance of the same-structured VLA in novel scenarios could boost from 14% to 69% with the guidance generated by the world model. Results demonstrate that our method outperforms previous baselines with a clear margin, particularly in out-of-distribution scenarios. Project page: \href{https://vista-wm.github.io/}{https://vista-wm.github.io}

机器人操作分层控制视觉规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。