用视觉反馈闭环优化机器人任务规划，让语言模型自动改错。
VersualRL: Closed-Loop Verbal Reinforcement Learning with Visual Execution Feedback for Task-Level Robot Planning
- 用语言模型分析视觉和执行日志，生成可理解的改进建议。
- 在真实机器人上实现多阶段任务，失败后自动调整策略。
- 无需训练模型，适合需要透明决策的工业级机器人应用。
我们提出 VersualRL，一种用于任务级机器人规划的闭环框架，通过视觉执行反馈迭代优化可执行的行为树（Behavior Tree），并利用结构化自然语言进行批判性反馈。该框架结合视觉-语言模型作为评判者，分析视觉观测与行为树执行轨迹；同时使用大语言模型作为执行者，完成离散且可解释的策略更新。在实际部署中，两个基础模型保持冻结，仅在符号层面更新行为树，不依赖在线梯度优化。这实现了透明推理、明确因果反馈和人类可读的策略演化。我们在真实移动机器人上验证了该框架在存在执行不确定性情况下的多阶段导航与操作任务表现。实验结果表明，该框架支持可解释的策略改进，能闭环适应执行失败，并成功部署于物理机器人系统。
原文摘要 · Abstract (English)
We introduce VersualRL, a closed-loop framework for task-level robot planning that uses visual execution feedback to iteratively refine executable Behavior Trees through structured natural-language critique. VersualRL combines a vision-language model critic, which analyzes visual observations and Behavior Tree execution traces, with a large language model actor that performs discrete and interpretable policy updates. During physical deployment, both foundation models remain frozen, while the executed Behavior Tree is updated at the symbolic level without online gradient-based policy optimization. This enables transparent reasoning, explicit causal feedback, and human-interpretable policy evolution. We validate VersualRL on a real mobile robot performing a multi-stage navigation and manipulation task under execution uncertainty. Experimental results show that the framework supports explainable policy improvements, closed-loop adaptation to execution failures, and successful deployment on physical robotic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。