让视觉语言动作模型通过在线搜索预见未来,解决长任务规划难题。
VLA-Reasoner: Empowering Vision-Language-Action Models with Reasoning via Online Monte Carlo Tree Search
- 用蒙特卡洛树搜索结合世界模型,滚动预测动作后果。
- 在仿真和真实环境中显著提升长程任务成功率,最高达92.3%。
- 适合需要长期推理的机器人操作场景,如复杂装配任务。
视觉-语言-动作模型(VLAs)通过模仿学习在通用机器人操作任务中表现优异,但现有方法仅能预测短视下一步动作,难以应对长时序轨迹任务,因累积偏差导致失败。为此,我们提出一种可插拔框架VLA-Reasoner,通过测试时扩展实现对未来的预见能力。该框架在世界模型支持下采样并展开可能的动作轨迹,利用视觉语言动作模型生成未来状态,从而进行结果推理与最优动作搜索。进一步引入蒙特卡洛树搜索(MCTS)提升大动作空间下的搜索效率,以逐步的VLA预测作为根节点初始化。同时,基于核密度估计(KDE)设计置信度采样机制,减少冗余的VLA调用,促进高效探索。采用离线价值评估策略,在MCTS中评估中间状态,为未来预测打分并利用长期反馈纠正偏差。我们在多个模拟环境和真实机器人上进行了大量实验,结果表明,VLA-Reasoner在多项指标上显著优于当前最先进方法,最高达成92.3%的成功率。本工作揭示了可扩展测试时计算在机器人操作中的潜力。
原文摘要 · Abstract (English)
Vision-Language-Action models (VLAs) achieve strong performance in general robotic manipulation tasks by scaling imitation learning. However, existing VLAs are limited to predicting short-sighted next-action, which struggle with long-horizon trajectory tasks due to incremental deviations. To address this problem, we propose a plug-in framework named \method that effectively empowers off-the-shelf VLAs with the capability of foreseeing future states via test-time scaling. Specifically, \method samples and rolls out possible action trajectories where involved actions are rationales to generate future states via a world model, which enables \method to foresee and reason potential outcomes and search for the optimal actions. We further leverage Monte Carlo Tree Search (MCTS) to improve search efficiency in large action spaces, where step-wise VLA predictions seed the root. Meanwhile, we introduce a confidence sampling mechanism based on Kernel Density Estimation (KDE), to enable efficient exploration in MCTS without redundant VLA queries. We evaluate intermediate states in MCTS via an offline value estimation strategy, to score predicted futures and correct deviations with long-term feedback. We conducted extensive experiments in both simulators and the real world, demonstrating that our proposed VLA-Reasoner achieves significant improvements over the state-of-the-art VLAs. Our method highlights a potential pathway toward scalable test-time computation of robotic manipulation. The project website is available at: https://vla-reasoner.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。