用逻辑编程构建强化学习抽象，让智能体更高效地学习复杂任务。
Answer-Set-Programming-based Abstractions for Reinforcement Learning
- 采用答案集编程（ASP）实现可读性强的领域抽象
- 在积木世界和迷你网格任务中验证了抽象效果
- 适合有先验知识的强化学习问题研究者
强化学习（RL）使智能体能从经验中学习策略，但现实问题常涉及巨大状态空间，导致学习与泛化困难。抽象与近似因此至关重要。关系强化学习（RRL）可通过对象及其关系进行推理，而CARCASS框架由Martijn van Otterlo提出，展示了如何用逻辑表示在一阶域中建模马尔可夫决策过程（MDP）。原版基于Prolog实现，利用领域知识生成强大抽象。本文探索答案集编程（ASP）——一种比Prolog更丰富且完全声明式的建模语言——来实现CARCASS抽象。我们在积木世界（Blocks World）和迷你网格（Minigrid）两个领域案例中评估该方法。结果表明，结合ASP的CARCASS为构建强化学习抽象提供了一条有前景的路径，尤其在具备领域知识时表现突出。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) enables autonomous agents to learn policies from experience, but realistic problems often involve enormous state spaces, making learning and generalisation challenging. Abstraction and approximation are therefore essential. Relational Reinforcement Learning (RRL) offers a way to reason about objects and their relations, and the CARCASS framework by Martijn van Otterlo demonstrates how logical representations can model Markov Decision Processes (MDPs) in first-order domains. Originally implemented in Prolog, CARCASS leverages domain knowledge to create powerful abstractions. We explore Answer-Set Programming (ASP), which is a rich and, contrary to Prolog, fully declarative modelling language, to realise CARCASS abstractions. We evaluate our ASP-based implementation in case studies of two domains, viz. Blocks World and Minigrid. Our results indicate that CARCASS with ASP provides a promising approach to constructing abstractions for RL, especially when domain knowledge is available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。