用大模型提升记忆效率，让智能体更聪明地学习。
Agentic Episodic Control
- 用大模型生成语义表示，替代传统浅层编码
- 只在关键状态召回经验，提升数据使用效率2-6倍
- 适合需要高效学习和跨任务适应的强化学习场景
强化学习（RL）仍受限于数据效率低下和泛化能力弱。以往的情景式强化学习方法通过外部记忆模块缓解此问题，但存在两个核心缺陷：浅层编码器导致表征瓶颈，以及记忆检索缺乏选择性。为此，我们提出一种新架构——代理式情景控制（AEC），将大语言模型（LLMs）融入情景式强化学习。AEC采用基于大模型的语义增强器，从原始观测中生成语义表示，并引入关键状态识别器，仅在必要时选择性召回有价值的经验。这使记忆使用从被动相似匹配转变为有策略、上下文感知的回忆。在五个BabyAI-Text环境中，AEC相较基线实现2-6倍更高的数据效率，且是唯一能在复杂任务UnlockLocal上达到90%以上成功率的方法。它还展现出强跨任务与跨环境泛化能力，在分布偏移下仍保持性能。结果表明，结合大模型先验与强化学习可显著提升智能体的样本效率与适应性。
原文摘要 · Abstract (English)
Reinforcement learning (RL) remains fundamentally limited by poor data efficiency and weak generalization. Prior episodic RL methods attempt to alleviate this via external memory modules, yet they suffer from two key limitations: a representation bottleneck caused by shallow encoders, and a retrieval dilemma where episodic memory is accessed indiscriminately. To address these challenges, we propose Agentic Episodic Control (AEC), a novel architecture that integrates large language models (LLMs) into episodic RL. AEC uses an LLM-based semantic augmenter to generate semantic representations from raw observations, and a critical state recognizer to selectively retrieve valuable experiences. This transforms memory usage from passive similarity matching into strategic, context-aware recall. Across five BabyAI-Text environments, AEC achieves 2-6x higher data efficiency than baselines and is the only method to solve complex tasks like UnlockLocal with over 90% success. It further demonstrates strong cross-task and cross-environment generalization, maintaining performance even under distribution shifts. AEC shows that combining LLM-derived priors with reinforcement learning yields more sample-efficient and adaptable agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。