让大模型像人一样思考并控制机械臂,实现闭环自主操作。
ALRM: Agentic LLM for Robotic Manipulation
- 用推理循环让大模型分步规划、执行、反思并修正动作。
- 在56个任务中验证,Claude-4.1-Opus和Falcon-H1-7B表现最优。
- 支持代码直接生成控制指令,适合需要可解释性的机器人项目。
大语言模型(LLMs)虽已展现出先进推理与规划能力,但在机器人控制中的应用仍受限于两点:一是现有基于LLM的方法缺乏模块化、闭环的智能体执行机制,难以实现持续规划与修正;二是现有操控任务基准侧重底层控制,未系统评估多步推理与语言多样性。本文提出面向机器人操控的智能体大模型(ALRM),通过类似ReAct的推理循环,将策略生成与智能体执行结合,支持两种模式:代码即策略(CaP)直接生成可执行控制代码,工具即策略(TaP)通过迭代规划与工具调用执行动作。为系统评估,我们构建了一个包含56个任务的新仿真基准,覆盖多个环境并涵盖语言多样指令。十种大模型实验表明,ALRM提供了一种可扩展、可解释且模块化的路径,连接自然语言推理与可靠机器人执行。结果揭示,在CaP模式下,Claude-4.1-Opus为最佳闭源模型,Falcon-H1-7B为最佳开源模型。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently empowered agentic frameworks to exhibit advanced reasoning and planning capabilities. However, their integration in robotic control pipelines remains limited in two aspects: (1) prior \ac{llm}-based approaches often lack modular, agentic execution mechanisms, limiting their ability to plan, reflect on outcomes, and revise actions in a closed-loop manner; and (2) existing benchmarks for manipulation tasks focus on low-level control and do not systematically evaluate multistep reasoning and linguistic variation. In this paper, we propose Agentic LLM for Robot Manipulation (ALRM), an LLM-driven agentic framework for robotic manipulation. ALRM integrates policy generation with agentic execution through a ReAct-style reasoning loop, supporting two complementary modes: Code-asPolicy (CaP) for direct executable control code generation, and Tool-as-Policy (TaP) for iterative planning and tool-based action execution. To enable systematic evaluation, we also introduce a novel simulation benchmark comprising 56 tasks across multiple environments, capturing linguistically diverse instructions. Experiments with ten LLMs demonstrate that ALRM provides a scalable, interpretable, and modular approach for bridging natural language reasoning with reliable robotic execution. Results reveal Claude-4.1-Opus as the top closed-source model and Falcon-H1-7B as the top open-source model under CaP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。