让大模型机器人学会从经验中学习，提升复杂任务成功率。
RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models
- 用记忆库自动调用过往经验作为上下文，辅助决策
- 在BabyAI和AlfWorld上任务成功率显著高于现有方法
- 适合需要长期学习与自我改进的智能体系统
大语言模型（LLMs）近年来成为解决复杂机器人任务的有力工具，即使在动作与观测存在不确定性的情况下也能表现良好。尽管基于LLM的决策方法（又称LLM代理）在搭配合适评价器后已展现出解决长周期复杂任务的潜力，但多数现有方法缺乏对过往交互经验的存储与学习能力——这是学习型机器人系统的关键特征。本文提出RAG-Modulo框架，通过引入历史交互记忆并结合评价器来评估代理决策。记忆模块可自动检索并融入相关过往经验作为上下文示例，为决策提供情境感知反馈；同时通过更新记忆，代理能持续优化性能，体现学习特性。在具有挑战性的BabyAI和AlfWorld环境中进行实验，结果表明该框架在任务成功率和效率方面均有显著提升，优于当前最先进基线方法。
原文摘要 · Abstract (English)
Large language models (LLMs) have recently emerged as promising tools for solving challenging robotic tasks, even in the presence of action and observation uncertainties. Recent LLM-based decision-making methods (also referred to as LLM-based agents), when paired with appropriate critics, have demonstrated potential in solving complex, long-horizon tasks with relatively few interactions. However, most existing LLM-based agents lack the ability to retain and learn from past interactions - an essential trait of learning-based robotic systems. We propose RAG-Modulo, a framework that enhances LLM-based agents with a memory of past interactions and incorporates critics to evaluate the agents' decisions. The memory component allows the agent to automatically retrieve and incorporate relevant past experiences as in-context examples, providing context-aware feedback for more informed decision-making. Further by updating its memory, the agent improves its performance over time, thereby exhibiting learning. Through experiments in the challenging BabyAI and AlfWorld domains, we demonstrate significant improvements in task success rates and efficiency, showing that the proposed RAG-Modulo framework outperforms state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。