用逻辑推理替代奖励学习,实现可验证的安全决策。
Constructive Symbolic Reinforcement Learning via Intuitionistic Logic and Goal-Chaining Inference
- 以直觉逻辑构建可证明的行动链,确保每步都有前提支持。
- 在网格世界中零无效动作,100% 安全且快速收敛。
- 适合需要可解释性与安全性的高风险场景如医疗、自动驾驶。
我们提出一种新型学习与规划框架,用构造性逻辑推理取代传统的基于奖励的优化。在该模型中,动作、状态转移和目标均表示为逻辑命题,决策过程通过直觉逻辑下的构造性证明进行。只有当状态转移和策略有可验证前提支持时才被接受,摒弃了概率性试错,确保逻辑有效性。我们实现了一个符号化智能体,在结构化网格世界中完成目标需满足一系列中间子目标(如收集钥匙开门),每个子目标受逻辑约束。与传统强化学习代理相比,无需大量探索,避免不安全或无效的转移。本方法通过目标链推理、条件追踪和知识积累,构建出可证明正确的计划。与Q-learning的实证对比显示,该方法实现了100%的安全性、可解释的行为和高效收敛,无任何无效动作,展现出在安全规划、符号认知和可信人工智能中的潜力。这项工作为强化学习开辟了新方向:不再依赖数值优化,而是基于构造性逻辑与证明理论。
原文摘要 · Abstract (English)
We introduce a novel learning and planning framework that replaces traditional reward-based optimisation with constructive logical inference. In our model, actions, transitions, and goals are represented as logical propositions, and decision-making proceeds by building constructive proofs under intuitionistic logic. This method ensures that state transitions and policies are accepted only when supported by verifiable preconditions -- eschewing probabilistic trial-and-error in favour of guaranteed logical validity. We implement a symbolic agent operating in a structured gridworld, where reaching a goal requires satisfying a chain of intermediate subgoals (e.g., collecting keys to open doors), each governed by logical constraints. Unlike conventional reinforcement learning agents, which require extensive exploration and suffer from unsafe or invalid transitions, our constructive agent builds a provably correct plan through goal chaining, condition tracking, and knowledge accumulation. Empirical comparison with Q-learning demonstrates that our method achieves perfect safety, interpretable behaviour, and efficient convergence with no invalid actions, highlighting its potential for safe planning, symbolic cognition, and trustworthy AI. This work presents a new direction for reinforcement learning grounded not in numeric optimisation, but in constructive logic and proof theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。