用LLM增强逻辑推理,让模型既懂知识又保持严谨
Sound and Complete Neurosymbolic Reasoning with LLM-Grounded Interpretations
- 将LLM嵌入非经典逻辑的解释函数,实现神经符号融合
- 在GPQA和SimpleQA上提升6个百分点的宏观F1值
- 适合需要高可信度推理的医疗、法律等安全场景
大语言模型在自然语言理解与生成方面表现优异,但在输出中常出现逻辑不一致问题。如何利用其广泛的知识进行形式化推理?本文提出一种方法,直接将LLM集成到一种次协调逻辑的形式语义解释函数中。通过基于短形式事实性基准数据集GPQA和SimpleQA的实证评估,双侧事实性评价相较单侧基线在两个基准上均提升约6个百分点的宏平均F1值(代价是覆盖率下降,因不一致或不确定情形会触发回避)。我们进一步实现了一个概念验证的表式推理机,并应用于包含228条断言和712条推导命题的药物安全知识库:系统检测出92个表示医学显著错误的矛盾(如推导出阿片类药物无成瘾性、β受体阻滞剂对哮喘安全),同时系统仍保持可满足性,证明矛盾被局部化而非引发逻辑爆炸。与以往工作不同,本方法提供理论框架与实际实现,使神经符号推理能利用LLM知识并保持底层逻辑的可靠性和完备性。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but exhibit problems with logical consistency in their output. How can we harness LLMs' broad-coverage parametric knowledge in formal reasoning despite their inconsistency? We present a method for directly integrating an LLM into the interpretation function of the formal semantics for a paraconsistent logic. We evaluate the method empirically using datasets derived from the short-form factuality benchmarks GPQA and SimpleQA, showing that bilateral factuality evaluation improves macro-F1 over a unilateral baseline by roughly 6 percentage points on both benchmarks (at the cost of reduced coverage, as abstention is triggered on inconsistent or uncertain cases). We further describe a proof-of-concept tableau reasoner implementing the method, and apply it to a medication-safety knowledge base of 228 asserted and 712 inferred statements: the system detects 92 gluts corresponding to medically significant errors (e.g., opioids inferred as non-addictive, beta-blockers inferred as safe in asthma) while remaining satisfiable, demonstrating that contradictions are localized rather than causing logical explosion. Unlike prior work, our method offers a theoretical framework with a practical implementation for neurosymbolic reasoning that leverages an LLM's knowledge while preserving the underlying logic's soundness and completeness properties.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。