让机器人模型学会思考:通过教师引导推理,提升复杂任务的执行成功率。
ReFineVLA: Reasoning-Aware Teacher-Guided Transfer Fine-Tuning
- 用专家模型生成动作推理理由,增强数据中的逻辑信息
- 在模拟机器人任务中,成功率平均提升5.0%至8.6%
- 适合需要可解释性与长程规划的机器人决策研究
视觉-语言-动作(VLA)模型因其能将多模态观测与语言指令转化为机器人动作而受到关注。然而,现有模型通常忽略显式推理过程,仅学习输入到动作的功能映射,削弱了对复杂、长时序操作任务的可解释性与泛化能力。本文提出ReFineVLA,一种基于教师引导推理的微调框架。首先,利用专家教师模型为机器人数据生成推理理由,增强数据语义;随后,使用这些富含推理信息的数据对预训练的VLA模型进行微调,同时保持其泛化能力并提升推理性能。我们通过注意力可视化分析发现,经训练的模型能更聚焦于相关物体与动作,显著改善跨任务理解。在多种设置下评估显示,ReFineVLA在SimplerEnv WidowX机器人任务中平均提升5.0%成功率,在变体聚合场景中提升8.6%,在视觉匹配场景中对Google Robot任务提升1.7%。代码将公开。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have gained much attention from the research community thanks to their strength in translating multimodal observations with linguistic instructions into robotic actions. Despite their recent advancements, VLAs often overlook the explicit reasoning and only learn the functional input-action mappings, omitting these crucial logical steps for interpretability and generalization for complex, long-horizon manipulation tasks. In this work, we propose \textit{ReFineVLA}, a multimodal reasoning-aware framework that fine-tunes VLAs with teacher-guided reasons. We first augment robotic datasets with reasoning rationales generated by an expert teacher model, guiding VLA models to learn to reason about their actions. Then, we use \textit{ReFineVLA} to fine-tune pre-trained VLAs with the reasoning-enriched datasets, while maintaining their inherent generalization abilities and boosting reasoning capabilities. In addition, we conduct an attention map visualization to analyze the alignment among visual attention, linguistic prompts, and to-be-executed actions of \textit{ReFineVLA}, showcasing its ability to focus on relevant tasks and actions. Through the latter step, we explore that \textit{ReFineVLA}-trained models exhibit a meaningful attention shift towards relevant objects, highlighting the enhanced multimodal understanding and improved generalization. Evaluated across manipulation tasks, \textit{ReFineVLA} outperforms the state-of-the-art baselines. Specifically, it achieves an average increase of $5.0\%$ success rate on SimplerEnv WidowX Robot tasks, improves by an average of $8.6\%$ in variant aggregation settings, and by $1.7\%$ in visual matching settings for SimplerEnv Google Robot tasks. The source code will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。