将符号推理融入深度强化学习，提升智能体决策能力
SymDQN: Symbolic Knowledge and Reasoning in Neural Network-based Reinforcement Learning
- 用逻辑张量网络模块增强DuelDQN，实现符号化控制
- 在5x5网格中使智能体准确识别形状并预测奖励，性能显著提升
- 适合研究神经符号融合的开发者，尤其关注可解释性强化学习
我们提出一种可在深度神经网络强化学习中引入符号控制与引导的学习架构。设计了SymDQN，一种基于逻辑张量网络（LTNs）的模块化方法，对现有双分支深度Q网络（DuelDQN）进行增强。该模块帮助智能体进行形状识别和奖励预测，指导动作策略学习，使其行为符合对环境的推理。实验为模块消融研究，在一个5x5网格环境中进行，智能体需导航并遇到不同形状，每种形状关联特定奖励。基础的DuelDQN尝试学习最优行为，而模块则辅助完成感知与预测任务。结果表明，该架构显著提升了学习性能与智能体行为精度。其模块化设计有助于深入理解神经与符号方法在强化学习中的融合复杂性。
原文摘要 · Abstract (English)
We propose a learning architecture that allows symbolic control and guidance in reinforcement learning with deep neural networks. We introduce SymDQN, a novel modular approach that augments the existing Dueling Deep Q-Networks (DuelDQN) architecture with modules based on the neuro-symbolic framework of Logic Tensor Networks (LTNs). The modules guide action policy learning and allow reinforcement learning agents to display behaviour consistent with reasoning about the environment. Our experiment is an ablation study performed on the modules. It is conducted in a reinforcement learning environment of a 5x5 grid navigated by an agent that encounters various shapes, each associated with a given reward. The underlying DuelDQN attempts to learn the optimal behaviour of the agent in this environment, while the modules facilitate shape recognition and reward prediction. We show that our architecture significantly improves learning, both in terms of performance and the precision of the agent. The modularity of SymDQN allows reflecting on the intricacies and complexities of combining neural and symbolic approaches in reinforcement learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。