让机器人能识别并拒绝不合理指令,提升真实场景下的操作鲁棒性。
RationalVLA: A Rational Vision-Language-Action Model with Dual System
- 采用双系统架构,融合视觉语言模型与低层控制策略,实现指令推理与执行
- 在14000+样本的缺陷指令数据集上,成功率高出基准14.5%,任务长度更优
- 适合需要理解复杂指令、应对模糊或错误命令的机器人应用
真实世界机器人部署的关键在于理解并响应自然语言指令。现有语言驱动的操作任务通常假设指令与环境完全对齐,这一假设限制了模型在现实场景中的鲁棒性和泛化能力,尤其当指令存在歧义、无关或不可行时。为此,我们提出RAtional MAnipulation (RAMA)基准,挑战模型处理未见过的有效指令及应被拒绝的缺陷指令。RAMA构建了一个包含超过14,000个样本的数据集,涵盖视觉、物理、语义、运动、安全和上下文外六类缺陷指令。我们进一步提出理性视觉-语言-动作模型(RationalVLA),其为机械臂设计的双系统结构,通过可学习的潜在空间嵌入将高层视觉-语言模型与底层操作策略融合。该设计使RationalVLA能够对指令进行推理,拒绝不可行命令,并有效执行操作。实验表明,RationalVLA在RAMA上的成功率比现有最佳基线高14.5%,平均任务长度达0.94,同时在标准操作任务上保持竞争力。真实世界测试进一步验证了其在实际应用中的有效性与鲁棒性。
原文摘要 · Abstract (English)
A fundamental requirement for real-world robotic deployment is the ability to understand and respond to natural language instructions. Existing language-conditioned manipulation tasks typically assume that instructions are perfectly aligned with the environment. This assumption limits robustness and generalization in realistic scenarios where instructions may be ambiguous, irrelevant, or infeasible. To address this problem, we introduce RAtional MAnipulation (RAMA), a new benchmark that challenges models with both unseen executable instructions and defective ones that should be rejected. In RAMA, we construct a dataset with over 14,000 samples, including diverse defective instructions spanning six dimensions: visual, physical, semantic, motion, safety, and out-of-context. We further propose the Rational Vision-Language-Action model (RationalVLA). It is a dual system for robotic arms that integrates the high-level vision-language model with the low-level manipulation policy by introducing learnable latent space embeddings. This design enables RationalVLA to reason over instructions, reject infeasible commands, and execute manipulation effectively. Experiments demonstrate that RationalVLA outperforms state-of-the-art baselines on RAMA by a 14.5% higher success rate and 0.94 average task length, while maintaining competitive performance on standard manipulation tasks. Real-world trials further validate its effectiveness and robustness in practical applications. Our project page is https://irpn-eai.github.io/RationalVLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。