让大模型动态选择逻辑求解器,提升推理准确率27%以上。
Adaptive LLM-Symbolic Reasoning via Dynamic Logical Solver Composition
- 大模型自动识别自然语言中的推理策略,动态调用对应逻辑求解器。
- 在多范式任务上相比GPT-4o提升27%,比DeepSeek-V3.1提升6%。
- 适用于需要灵活推理的复杂任务,尤其适合想提升大模型逻辑能力的研究者。
神经符号自然语言处理方法旨在结合大语言模型与形式化逻辑求解器的互补优势。然而,现有方法多为静态集成,目标求解器在设计时即固定,难以适应多样化的形式化推理策略。为此,本文提出一种自适应、多范式神经符号推理框架:(1) 自动从自然语言问题中识别出所需的形式化推理策略;(2) 通过自动形式化接口动态选择并应用专用逻辑求解器。在单任务与多范式推理任务上的大量实验表明:大模型预测必要推理策略的准确率超过90%。该机制实现灵活的逻辑求解器集成,使本框架在性能上分别优于GPT-4o和DeepSeek-V3.1达27%和6%。此外,自适应推理还能正向提升纯大模型方法,在零样本、思维链(CoT)和符号思维链(symbolic CoT)设置下,对GPT-4o分别带来10%、5%和6%的提升。尽管小模型在自适应神经符号推理中表现较弱,但后训练可提供可行的改进路径。整体而言,本工作奠定了自适应大模型-符号推理的基础,为统一处理异构推理挑战提供了新方向。
原文摘要 · Abstract (English)
Neuro-symbolic NLP methods aim to leverage the complementary strengths of large language models and formal logical solvers. However, current approaches are mostly static in nature, i.e., the integration of a target solver is predetermined at design time, hindering the ability to employ diverse formal inference strategies. To address this, we introduce an adaptive, multi-paradigm, neuro-symbolic inference framework that: (1) automatically identifies formal reasoning strategies from problems expressed in natural language; and (2) dynamically selects and applies specialized formal logical solvers via autoformalization interfaces. Extensive experiments on individual and multi-paradigm reasoning tasks support the following conclusions: LLMs are effective at predicting the necessary formal reasoning strategies with an accuracy above 90 percent. This enables flexible integration with formal logical solvers, resulting in our framework outperforming competing baselines by 27 percent and 6 percent compared to GPT-4o and DeepSeek-V3.1, respectively. Moreover, adaptive reasoning can even positively impact pure LLM methods, yielding gains of 10, 5, and 6 percent on zero-shot, CoT, and symbolic CoT settings with GPT-4o. Finally, although smaller models struggle with adaptive neuro-symbolic reasoning, post-training offers a viable path to improvement. Overall, this work establishes the foundations for adaptive LLM-symbolic reasoning, offering a path forward for unifying material and formal inferences on heterogeneous reasoning challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。