用双模型分层解析自然语言指令，让机器人更懂人话地完成复杂任务。
Hierarchical Prompting with Dual LLM Modules for Robotic Task and Motion Planning
- 高阶规划与低阶空间推理双模块协同，分别处理指令与精准定位。
- 在24个场景中达成86%的任务成功率，对复杂指令响应稳定。
- 适合需要自然语言交互的家居服务机器人研发与应用。
我们提出一种分层式语言驱动框架，用于提升服务与辅助场景中人机交互的自然性与直观性。系统采用两个大语言模型模块：高层规划代理负责解析自然语言指令，生成动作序列，使用类似ReAct的提示策略，并调用工具实现物体感知与操作（如抓取、放置、释放）；针对精确空间定位（如“把杯子放在盘子旁边”），另设专用子模块进行基于物体几何与场景布局的3D推理。系统集成YOLOX-GDRNet用于物体检测与位姿估计，并配备运动执行模拟器。在24个测试场景中评估，涵盖从简单空间指令到高层级指令及不可行请求，整体任务成功率达到86%。
原文摘要 · Abstract (English)
We present a hierarchical language-driven framework for robotic task and motion planning to improve natural, intuitive human-robot interaction in service and assistance scenarios. The proposed system employs two large language model (LLM) modules: a high-level planning agent and a low-level spatial reasoning sub-module. The primary agent processes natural language commands and generates action sequences using a ReAct-style prompt, interacting with tools for object perception and manipulation (e.g., pick, place, release). For precise spatial placement, such as interpreting "place the mug next to the plate", a separate sub-prompting module handles 3D reasoning based on object geometry and scene layout. The system integrates YOLOX-GDRNet for object detection and pose estimation, along with a motion execution stub. We evaluated the system in 24 test scenarios, ranging from simple spatial commands to high-level instructions and infeasible requests. The system achieved an overall task success rate of 86%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。