用逻辑规则在线指导强化学习探索，提升效率并保持可解释性。
Online inductive learning from answer sets for efficient reinforcement learning exploration
- 从经验数据中学习逻辑规则，动态生成可解释的策略近似。
- 在首个训练批次即显著提升累积回报，且不增加计算开销。
- 适合需要高效探索与透明决策过程的强化学习应用。
本文提出一种结合归纳逻辑编程与强化学习的新方法，以提升训练性能并增强可解释性。通过从噪声数据中学习答案集程序，获取一组逻辑规则来近似代理在每批经验下的策略。随后对学习到的规则进行答案集推理，指导下一阶段的探索行为，无需低效的奖励塑造，同时保留最优性并施加软偏差。整个流程在强化学习在线执行期间完成。我们在两个复杂度递增的地图上将该方法集成至Q-learning算法，用于Pac-Man场景。结果表明，该方法在训练初期即显著提升累积回报；归纳学习未增加计算时间，且所学规则迅速收敛为代理策略的合理解释。
原文摘要 · Abstract (English)
This paper presents a novel approach combining inductive logic programming with reinforcement learning to improve training performance and explainability. We exploit inductive learning of answer set programs from noisy examples to learn a set of logical rules representing an explainable approximation of the agent policy at each batch of experience. We then perform answer set reasoning on the learned rules to guide the exploration of the learning agent at the next batch, without requiring inefficient reward shaping and preserving optimality with soft bias. The entire procedure is conducted during the online execution of the reinforcement learning algorithm. We preliminarily validate the efficacy of our approach by integrating it into the Q-learning algorithm for the Pac-Man scenario in two maps of increasing complexity. Our methodology produces a significant boost in the discounted return achieved by the agent, even in the first batches of training. Moreover, inductive learning does not compromise the computational time required by Q-learning and learned rules quickly converge to an explanation of the agent policy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。