让智能体自动学习可复用的抽象能力，提升复杂任务规划效率。
Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents
- 利用大模型上下文学习能力，从经验中主动构建抽象知识
- 在多个环境上实现显著更少样本下的任务求解成功率
- 仅需少量人工提示，适合需要高效泛化的程序合成场景
人类通过学习抽象概念来高效规划并快速泛化到新任务，这一能力对当前最先进的大语言模型（LLM）代理和深度强化学习（RL）系统仍是挑战。受认知科学启发，基于理论的强化学习（TBRL）系统如TheoryCoder能通过有效使用抽象实现强泛化，但严重依赖人工提供的抽象，回避了抽象学习问题。我们提出TheoryCoder-2，一种新型TBRL代理，利用大模型的上下文学习能力，从经验中合成可复用的抽象，并将其融入层次化规划过程。我们在多种环境（包括BabyAI、Minihack和Sokoban等VGDL游戏）上进行实验，发现TheoryCoder-2相比基线方法（如经典规划域构建、基于推理的规划及WorldCoder等程序合成代理）具有显著更高的样本效率，能够解决基线失败的复杂任务，且仅需极少量人工提示，优于以往的TBRL系统。
原文摘要 · Abstract (English)
Humans learn abstractions and use them to plan efficiently to quickly generalize across tasks -- an ability that remains challenging for state-of-the-art large language model (LLM) agents and deep reinforcement learning (RL) systems. Inspired by the cognitive science of how people form abstractions and intuitive theories of their world knowledge, Theory-Based RL (TBRL) systems, such as TheoryCoder, exhibit strong generalization through effective use of abstractions. However, they heavily rely on human-provided abstractions and sidestep the abstraction-learning problem. We introduce TheoryCoder-2, a new TBRL agent that leverages LLMs' in-context learning ability to actively learn reusable abstractions rather than relying on hand-specified ones, by synthesizing abstractions from experience and integrating them into a hierarchical planning process. We conduct experiments on diverse environments, including BabyAI, Minihack and VGDL games like Sokoban. We find that TheoryCoder-2 is significantly more sample-efficient than baseline LLM agents augmented with classical planning domain construction, reasoning-based planning, and prior program-synthesis agents such as WorldCoder. TheoryCoder-2 is able to solve complex tasks that the baselines fail, while only requiring minimal human prompts, unlike prior TBRL systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。