将符号推理与神经策略融合，提升强化学习的适应性与可解释性。
BlendRL: A Framework for Merging Symbolic and Neural Policy Learning
- 用混合逻辑与神经策略构建统一框架，实现双范式协同。
- 在标准Atari环境中超越纯神经与纯符号基线表现。
- 适合需要高可靠性与可解释性的智能体应用。
人类能同时运用符号推理和直觉反应，而现有强化学习策略通常仅采用黑箱神经网络或依赖预设符号与规则的符号系统。这种割裂方式严重限制了智能体能力，使其要么缺乏神经智能体的灵活低级反应，要么缺少符号智能体的可解释性推理。为此，我们提出BlendRL——一种神经符号强化学习框架，将逻辑与神经策略有机结合于同一智能体中。实验证明，BlendRL智能体在标准Atari环境中的表现优于纯神经与纯符号基线，并对环境变化具有更强鲁棒性。进一步分析显示，两类策略的协同作用有效克服了彼此局限。
原文摘要 · Abstract (English)
Humans can leverage both symbolic reasoning and intuitive reactions. In contrast, reinforcement learning policies are typically encoded in either opaque systems like neural networks or symbolic systems that rely on predefined symbols and rules. This disjointed approach severely limits the agents' capabilities, as they often lack either the flexible low-level reaction characteristic of neural agents or the interpretable reasoning of symbolic agents. To overcome this challenge, we introduce BlendRL, a neuro-symbolic RL framework that harmoniously integrates both paradigms within RL agents that use mixtures of both logic and neural policies. We empirically demonstrate that BlendRL agents outperform both neural and symbolic baselines in standard Atari environments, and showcase their robustness to environmental changes. Additionally, we analyze the interaction between neural and symbolic policies, illustrating how their hybrid use helps agents overcome each other's limitations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。