用大模型让机器人记住过去动作，更聪明地完成家务。
LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics
- 三类专用大模型代理协同工作，实现任务规划与记忆追踪。
- 引入RAG机制后，长期物品追踪准确率显著提升。
- 适合研究家庭机器人、具身智能与大模型应用的开发者。
我们提出一种基于大语言模型驱动的具身机器人系统，用于自主家庭物品管理。该系统融合记忆增强型任务规划，使机器人在执行高层用户指令的同时可追踪历史行为。系统包含三个专用代理：路由代理、任务规划代理和知识库代理，均由针对任务优化的大模型驱动。通过上下文学习，系统无需显式训练即可适应新任务。RAG机制使系统能从过往交互中检索上下文，提升长期物品追踪能力。结合Grounded SAM与LLaMa3.2-Vision，实现鲁棒的对象检测，支持语义场景理解以辅助任务规划。在三个家庭场景中的评估表明，系统具备高任务规划准确率，并因RAG提升了记忆召回性能。具体而言，Qwen2.5在专用代理中表现最优，而LLaMA3.1在路由任务中更优。源代码已开源：https://github.com/marc1198/chat-hsr。
原文摘要 · Abstract (English)
We present an embodied robotic system with an LLM-driven agent-orchestration architecture for autonomous household object management. The system integrates memory-augmented task planning, enabling robots to execute high-level user commands while tracking past actions. It employs three specialized agents: a routing agent, a task planning agent, and a knowledge base agent, each powered by task-specific LLMs. By leveraging in-context learning, our system avoids the need for explicit model training. RAG enables the system to retrieve context from past interactions, enhancing long-term object tracking. A combination of Grounded SAM and LLaMa3.2-Vision provides robust object detection, facilitating semantic scene understanding for task planning. Evaluation across three household scenarios demonstrates high task planning accuracy and an improvement in memory recall due to RAG. Specifically, Qwen2.5 yields best performance for specialized agents, while LLaMA3.1 excels in routing tasks. The source code is available at: https://github.com/marc1198/chat-hsr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。