让大模型自动选自然语言或符号语言推理，提升数学逻辑题解答能力
HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning
- 根据问题特点动态选择自然语言或符号语言推理
- 在FOLIO和MATH数据集上比GPT-4o提升最高10%
- 适合需要精准推理的数学、逻辑类任务研究者
大语言模型通过自然语言或符号语言进行逻辑与数学推理。自然语言虽具人类可读性但存在歧义，符号推理则精度高且可机器执行，但受限于严格领域约束。本文提出HYBRIDMIND，一种自适应策略，为每个推理问题选择最优方法。通过大量实验，评估了基于提示的先进LLM与微调开源模型的表现。结果表明，将LLaMA-3.1-8B-Instruct微调为元选择器，在FOLIO和MATH数据集上分别比GPT-4o自然语言推理高出4.4%和1.3%。更显著的是，使用GPT-3.5-turbo作为提示型元选择器，在FOLIO难题子集上相较GPT-4o提升10%。代码与数据将公开，以支持后续研究。
原文摘要 · Abstract (English)
LLMs approach logical and mathematical reasoning through natural or symbolic languages. While natural language offers human-accessible flexibility but suffers from ambiguity, symbolic reasoning provides precise, machine-executable inferences at the cost of strict domain constraints. We introduce HYBRIDMIND, an adaptive strategy that selects the optimal reasoning approach for each reasoning problem. Through extensive experiments, we evaluate both prompting-based approaches with state-of-the-art LLMs and fine-tuned open-source models. We find that fine-tuning LLaMA-3.1-8B-Instruct as a meta-selector outperforms GPT-4o's natural language reasoning by 4.4\% on FOLIO and 1.3\% on MATH. More notably, using GPT-3.5-turbo as a prompted meta-selector yields a 10\% improvement on FOLIO's challenging subset compared to GPT-4o. We will release our code and data to support future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。