对比两种神经符号方法,发现混合式更利于通用逻辑推理。
A Comparative Study of Neurosymbolic AI Approaches to Interpretable Logical Reasoning
- 将符号推理嵌入神经网络或独立调用符号求解器
- 混合式在通用逻辑任务上表现更好且推理过程更透明
- 适合希望提升可解释性的研究者与应用开发者
通用逻辑推理,即在无特定领域任务中进行演绎推理的能力,仍是大语言模型(LLMs)的挑战。当前的LLMs无法实现确定性推理且缺乏可解释性。因此,神经符号人工智能(neurosymbolic AI)兴起,试图将逻辑引入神经网络。本文识别出两种主要方法:(i) 整合式,将符号推理包含在神经网络内部;(ii) 混合式,由独立于神经网络的符号求解器执行符号推理。尽管两者在特定领域基准上表现良好,但其在通用领域上的表现尚不明确。本文以代表性模型Logic Neural Network(LNN,整合式)和LLM-Symbolic Solver(LLM-SS,混合式)为案例,分析表明混合式更适用于发展通用逻辑推理,原因在于其推理链更具可解释性,并能保留现有LLMs的优势。为此,本文提出一个模块化、模型无关、领域无关且几乎无需人工干预的通用框架,支持未来研究。
原文摘要 · Abstract (English)
General logical reasoning, defined as the ability to reason deductively on domain-agnostic tasks, continues to be a challenge for large language models (LLMs). Current LLMs fail to reason deterministically and are not interpretable. As such, there has been a recent surge in interest in neurosymbolic AI, which attempts to incorporate logic into neural networks. We first identify two main neurosymbolic approaches to improving logical reasoning: (i) the integrative approach comprising models where symbolic reasoning is contained within the neural network, and (ii) the hybrid approach comprising models where a symbolic solver, separate from the neural network, performs symbolic reasoning. Both contain AI systems with promising results on domain-specific logical reasoning benchmarks. However, their performance on domain-agnostic benchmarks is understudied. To the best of our knowledge, there has not been a comparison of the contrasting approaches that answers the following question: Which approach is more promising for developing general logical reasoning? To analyze their potential, the following best-in-class domain-agnostic models are introduced: Logic Neural Network (LNN), which uses the integrative approach, and LLM-Symbolic Solver (LLM-SS), which uses the hybrid approach. Using both models as case studies and representatives of each approach, our analysis demonstrates that the hybrid approach is more promising for developing general logical reasoning because (i) its reasoning chain is more interpretable, and (ii) it retains the capabilities and advantages of existing LLMs. To support future works using the hybrid approach, we propose a generalizable framework based on LLM-SS that is modular by design, model-agnostic, domain-agnostic, and requires little to no human input.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。