选对形式语言能显著提升大模型的逻辑推理能力
Intermediate Languages Matter: Formal Languages and LLMs affect Neurosymbolic Reasoning
- 用形式语言作为中间层，让大模型翻译自然语言并交由符号求解器处理
- 不同形式语言使推理准确率差异达15%以上，影响语法与语义理解
- 适合研究大模型推理机制或开发智能系统的人参考
大语言模型在众多任务上表现惊人，但其形式化推理能力仍不足。神经符号推理是一种有前景的解决方案：利用大模型将自然语言翻译为形式语言，再由符号求解器生成正确结果。然而，该方法成功的关键因素尚不明确。本文揭示了一个被忽视的因素——形式语言的选择。我们提出中间语言挑战：为神经符号推理选择合适的正式语言。通过在三个数据集和七种大模型上对比四种形式语言，发现语言选择显著影响语法与语义推理能力，并在不同模型间产生差异性影响。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve astonishing results on a wide range of tasks. However, their formal reasoning ability still lags behind. A promising approach is Neurosymbolic LLM reasoning. It works by using LLMs as translators from natural to formal languages and symbolic solvers for deriving correct results. Still, the contributing factors to the success of Neurosymbolic LLM reasoning remain unclear. This paper demonstrates that one previously overlooked factor is the choice of the formal language. We introduce the intermediate language challenge: selecting a suitable formal language for neurosymbolic reasoning. By comparing four formal languages across three datasets and seven LLMs, we show that the choice of formal language affects both syntactic and semantic reasoning capabilities. We also discuss the varying effects across different LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。