arXiv:2409.17539cs.CL2024-09NAACL被引 36

通过逻辑扩展增强上下文,让大模型推理更准确可靠。

Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models

  • 用命题逻辑扩展原始输入,补全推理信息
  • 在5个逻辑任务上显著提升现有提示方法效果
  • 特别适合需要严格逻辑链的任务,如考试题解

大型语言模型在多项任务中表现卓越,但在复杂逻辑推理任务上仍不理想。尽管链式思维等提示方法有所改善,但存在结论与推理过程不一致的问题。已有研究尝试使用命题逻辑提升推理能力,但逻辑表达提取不全会导致信息丢失,引发错误结果。为此,我们提出逻辑思维(Logic-of-Thought, LoT)提示方法,利用命题逻辑生成扩展的逻辑信息描述,并作为额外补充注入原始上下文,确保信息完整,提升逻辑推理能力。LoT与现有提示方法正交,可无缝集成。大量实验表明,LoT在五个逻辑推理任务上显著提升多种提示方法的表现:在ReClor数据集上使链式思维提升4.35%,在RuleTaker数据集上使链式思维+自一致性提升3.52%,在ProofWriter数据集上使思维树提升8%。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks but their performance in complex logical reasoning tasks remains unsatisfactory. Although some prompting methods, such as Chain-of-Thought, can improve the reasoning ability of LLMs to some extent, they suffer from an unfaithful issue where derived conclusions may not align with the generated reasoning chain. To address this issue, some studies employ the approach of propositional logic to further enhance logical reasoning abilities of LLMs. However, the potential omissions in the extraction of logical expressions in these methods can cause information loss in the logical reasoning process, thereby generating incorrect results. To this end, we propose Logic-of-Thought (LoT) prompting which employs propositional logic to generate expanded logical information descriptions and utilizes them as an additional augmentation to original contexts, thereby ensuring information completeness and enhancing logical reasoning ability. LoT is orthogonal to existing prompting methods and can be seamlessly integrated with them. Extensive experiments demonstrate that LoT boosts the performance of various prompting methods with a striking margin across five logical reasoning tasks. In particular, LoT enhances Chain-of-Thought's performance on the ReClor dataset by +4.35%, improves Chain-of-Thought with Self-Consistency's performance on the RuleTaker dataset by +3.52%, and boosts performance of Tree-of-Thoughts on the ProofWriter dataset by +8%.

逻辑推理提示工程大模型命题逻辑

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。