用视觉语言动作模型预测价值,提升机器人长任务规划效果。
V-VLAPS: Value-Guided Planning for Vision-Language-Action Models
- 在树搜索中引入轻量级价值头,基于历史回放预测回报。
- 在五套LIBERO任务上,大预算下成功率提升4%~6%。
- 适合需要长期规划的机器人操作场景,尤其应对分布外情况。
视觉-语言-动作(VLA)模型为机器人操作提供强动作先验,但其反应式行为在分布外和长时序任务中易失效。现有方法虽利用预训练策略引导树搜索,但节点选择仍依赖策略先验和访问次数探索。当策略偏好错误动作时,规划器缺乏学习到的价值信号纠正偏差。先前研究发现VLA表示可编码回放成功与失败信息,提示其可用于规划中的价值估计。本文提出价值引导的视觉-语言-动作规划与搜索(V-VLAPS),通过离线VLA回放训练轻量价值头,预测蒙特卡洛回报,指导模拟中的蒙特卡洛树搜索向高价值分支推进。在五个LIBERO套件上,V-VLAPS在默认搜索预算下整体表现持平于无价值基线;分析显示,多数困难失败源于根节点超时,此时预测价值区分度弱。在更大搜索预算下,所有任务套件均超越基线,其中LIBERO-Object提升6个百分点,LIBERO-10提升4个百分点。结果表明,当搜索深入至价值排序关键分支时,VLA表示不仅支持失败预测,还可实现有效价值引导规划。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models provide strong action priors for robotic manipulation, but their reactive behavior can fail under distribution shift and long-horizon task structure. Recent VLA-guided planning methods improve execution by using pretrained policies to guide tree search, yet node selection still depends heavily on policy priors and visit-count exploration. Consequently, when the policy favors poor actions, the planner lacks a learned value signal to correct this bias. Prior work has shown that VLA representations encode rollout success and failure information, suggesting that they may also support value estimation during planning. We introduce Value-Guided Vision-Language-Action Planning and Search (V-VLAPS), which augments VLA-guided planning with a lightweight value head trained on offline VLA rollouts to predict Monte Carlo returns. These predictions guide Monte Carlo Tree Search in simulation toward higher-value branches. Across five LIBERO suites, V-VLAPS matches value-free planning baseline at the default search budget in aggregate, and analysis shows that many hard failures are root-level timeouts where predicted values are weakly separated. With a larger search budget, V-VLAPS improves over the baseline in all task suites with +6 percentage points on LIBERO-Object and +4 percentage points on LIBERO-10. Our results suggest that VLA representations can support not only failure prediction, but also value-guided planning when search reaches branches where value-based ranking matters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。