用检索增强生成让机器人听懂指令并安全执行抓取任务
ARRC: Advanced Reasoning Robot Control - Knowledge-Driven Autonomous Manipulation Using Retrieval-Augmented Generation
- 用RAG检索知识库，让大模型生成可执行的行动规划
- 在xArm 850上实现95%以上任务成功率，支持动态避障
- 适合做智能机器人交互与自主操作研究的开发者
我们提出ARRC（Advanced Reasoning Robot Control），一个将自然语言指令转化为安全本地控制的实用系统。该系统结合检索增强生成（RAG）、RGB-D感知与受保护执行，部署于低成本机械臂。系统将机器人知识（运动模式、任务模板、安全规则）索引至向量数据库，根据指令检索相关上下文，驱动大语言模型（LLM）生成JSON结构化动作计划。执行平台为配备Dynamixel驱动平行夹持器与Intel RealSense D435相机的UFactory xArm 850。感知通过AprilTag与深度信息融合，获得以物体为中心的度量位姿。执行通过软件安全门控保障：工作空间限制、速度/力矩上限、超时与有限重试机制。文中详述架构设计、知识构建、集成策略及可复现的桌面扫描、接近与抓放任务评估协议。实验表明该方法显著提升规划有效性与适应性，同时保持感知与底层控制的本地化。
原文摘要 · Abstract (English)
We present ARRC (Advanced Reasoning Robot Control), a practical system that connects natural-language instructions to safe local robotic control by combining Retrieval-Augmented Generation (RAG) with RGB-D perception and guarded execution on an affordable robot arm. The system indexes curated robot knowledge (movement patterns, task templates, and safety heuristics) in a vector database, retrieves task-relevant context for each instruction, and conditions a large language model (LLM) to produce JSON-structured action plans. Plans are executed on a UFactory xArm 850 fitted with a Dynamixel-driven parallel gripper and an Intel RealSense D435 camera. Perception uses AprilTag detections fused with depth to produce object-centric metric poses. Execution is enforced via software safety gates: workspace bounds, speed and force caps, timeouts, and bounded retries. We describe the architecture, knowledge design, integration choices, and a reproducible evaluation protocol for tabletop scan, approach, and pick-place tasks. Experimental results demonstrate the efficacy of the proposed approach. Our design shows that RAG-based planning can substantially improve plan validity and adaptability while keeping perception and low-level control local to the robot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。