让智能体边探索边更新知识，提升长程决策的样本效率。
Neurosymbolic Reasoning with Incremental Knowledge for Sample Efficient Hierarchical Reinforcement Learning
- 用可更新的知识表示支持符号规划与神经模块协同
- 在导航任务中样本效率提升显著，优于固定知识结构
- 适合需要持续学习与推理的复杂决策场景
传统强化学习在稀疏奖励、需长程推理的环境中表现不佳。标准分层强化学习（HRL）将知识以固定形式编码，无法随探索更新，导致初期知识不足时效率低下。本文提出神经符号分层强化学习（InK），通过可更新的增量知识表示，使高层符号组件使用 $D^*$ 等算法进行符号规划，底层目标条件神经模块则通过奖励塑形学习运动基元。实验表明，引入增量知识可显著提升样本效率。此外，为基于先验知识进行最优符号规划，我们构建了信念世界树搜索（Belief World Tree Search）。代码已开源：https://github.com/CPS-research-group/ink_bwts。
原文摘要 · Abstract (English)
(Flat) Reinforcement Learning (RL) agents face significant challenges in environments with sparse rewards that require long-horizon reasoning. A compelling approach to improve sample efficiency is to incorporate knowledge into learning and decision-making. In standard Hierarchical RL (HRL), knowledge is encoded in a fixed, non-updatable form, such as architectural choices, and remains unchanged throughout learning. With fixed HRL, reasoning with incremental knowledge learned during exploration is impractical before sufficient environmental knowledge is acquired, leading to poor sample efficiency. In this work, we propose neurosymbolic HRL with {\em Incremental Knowledge (InK)}: symbolic high-level components perform {\em symbolic planning} (e.g. using $D^*$) on an updatable representation of current InK, while low-level goal-conditioned neural modules learn motion primitives through experience using reward shaping. Experiments on navigation tasks demonstrate that incorporating InK substantially improves sample efficiency. Additionally, to perform {\em optimal} symbolic planning given {\em prior} knowledge about the world, we develop Belief World Tree Search. The code is available at https://github.com/CPS-research-group/ink_bwts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。