arXiv:2511.07910cs.CL2025-11被引 3

让大模型输出更合逻辑,专治知识推理中的自相矛盾

Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning

  • 从生成时的输出概率出发,动态修正逻辑错误
  • 在多个知识图谱问答任务上显著提升逻辑一致性
  • 无需复杂提示设计,适配不同任务和知识图谱

大语言模型(LLMs)通过海量无结构文本预训练,在自然语言推理中表现优异,能理解语义逻辑并生成一致回应。然而,无结构与结构化知识间的表征差异导致模型在知识图谱问答(KGQA)等任务中出现逻辑漂移问题。现有方法依赖复杂提示引导推理,仅提供输入层面指导,无法根本解决输出逻辑不一致问题,且流程僵化难适配不同任务与知识图谱。为此,我们提出「Logits-to-Logic」框架,聚焦自回归生成过程中的输出概率(logits),引入概率强化与过滤模块,主动修复生成结果中的逻辑缺陷。大量实验表明,该方法显著提升模型在结构化知识推理中的逻辑一致性,在多个KGQA基准上达到领先性能。

原文摘要 · Abstract (English)

Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enabling them to understand the logic in natural language and generate logic-consistent responses. However, the representational differences between unstructured and structured knowledge make LLMs inherently struggle to maintain logic consistency, leading to \textit{Logic Drift} challenges in structured knowledge reasoning tasks such as Knowledge Graph Question Answering (KGQA). Existing methods address this limitation by designing complex workflows embedded in prompts to guide LLM reasoning. Nevertheless, these approaches only provide input-level guidance and fail to fundamentally address the \textit{Logic Drift} in LLM outputs. Additionally, their inflexible reasoning workflows cannot adapt to different tasks and knowledge graphs. To enhance LLMs' logic consistency in structured knowledge reasoning, we specifically target the logits output from the autoregressive generation process. We propose the \textit{Logits-to-Logic} framework, which incorporates logits strengthening and logits filtering as core modules to correct logical defects in LLM outputs. Extensive experiments show that our approach significantly improves LLMs' logic consistency in structured knowledge reasoning and achieves state-of-the-art performance on multiple KGQA benchmarks.

逻辑推理知识图谱大模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。