arXiv:2505.12744cs.AI2025-05被引 6

让大模型直接推理机器人抓取动作,提升泛化与可解释性。

Incentivizing Multimodal Reasoning in Large Models for Direct Robot Manipulation

  • 用新轴表示法替代欧拉角,更好支持空间推理
  • 构建高质量对话数据集并强化学习,提升任务解决能力
  • 7B模型实现跨环境泛化,适合需要透明决策的场景

近期大型多模态模型展现出强大的推理能力,尤其在解决复杂数学问题和精准空间感知方面。本文核心观点是:这些能力可自然延伸至机器人操作,使大模型通过推理直接生成语言形式的下一步目标,而非依赖独立的动作头。然而该范式面临两大挑战:一是如何让大模型理解空间动作空间,二是如何充分挖掘其推理潜力。针对前者,提出新任务设定,输入物体部件与夹爪当前状态,并以新轴表示法替代传统欧拉角,该表示更契合空间推理且易于在统一语言空间中解释。针对后者,设计流水线,利用前沿大模型生成高质量、多轮对话的推理数据集,用于监督微调;随后在仿真环境中通过试错强化学习进一步提升模型的推理能力。基于7B模型构建的ReasonManip模型展现出三大优势:一是对分布外环境、物体和任务具有优异泛化性;二是得益于统一语言表示,具备天然的仿真到现实迁移能力;三是高阶推理与低阶控制之间具有透明可解释性。大量实验验证了该范式的有效性及其推动大模型驱动机器人操作的潜力。

原文摘要 · Abstract (English)

Recent Large Multimodal Models have demonstrated remarkable reasoning capabilities, especially in solving complex mathematical problems and realizing accurate spatial perception. Our key insight is that these emerging abilities can naturally extend to robotic manipulation by enabling LMMs to directly infer the next goal in language via reasoning, rather than relying on a separate action head. However, this paradigm meets two main challenges: i) How to make LMMs understand the spatial action space, and ii) How to fully exploit the reasoning capacity of LMMs in solving these tasks. To tackle the former challenge, we propose a novel task formulation, which inputs the current states of object parts and the gripper, and reformulates rotation by a new axis representation instead of traditional Euler angles. This representation is more compatible with spatial reasoning and easier to interpret within a unified language space. For the latter challenge, we design a pipeline to utilize cutting-edge LMMs to generate a small but high-quality reasoning dataset of multi-round dialogues that successfully solve manipulation tasks for supervised fine-tuning. Then, we perform reinforcement learning by trial-and-error interactions in simulation to further enhance the model's reasoning abilities for robotic manipulation. Our resulting reasoning model built upon a 7B backbone, named ReasonManip, demonstrates three notable advantages driven by its system-2 level reasoning capabilities: i) exceptional generalizability to out-of-distribution environments, objects, and tasks; ii) inherent sim-to-real transfer ability enabled by the unified language representation shared across domains; iii) transparent interpretability connecting high-level reasoning and low-level control. Extensive experiments demonstrate the effectiveness of the proposed paradigm and its potential to advance LMM-driven robotic manipulation.

多模态推理机器人操作语言空间强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。