用弱模型的多种看法提升强模型数学推理能力
Weaker LLMs' Opinions Also Matter: Mixture of Opinions Enhances LLM's Mathematical Reasoning
- 让多个弱模型提供不同解题思路，增强主模型学习
- 在GSM8K上平均提升5%准确率，优于传统微调方法
- 适合想低成本提升模型推理能力的研究者
大语言模型（LLMs）在数学推理方面取得进展，但小到中等规模的开源模型性能仍存疑。为此，我们提出一种后训练方法——混合观点（MoO），利用多个较弱的辅助模型生成链式思维（CoT）和答案，帮助较强主模型从多元视角学习。在数学推理基准测试中，该方法相比标准监督微调（SFT）、少样本提示和混合代理（MoA）平均提升5%准确率，验证了弱模型观点对强化推理的有效性。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) have raised interest in their formal reasoning capabilities, particularly in mathematics. While closed LLMs like GPT-4 perform well on mathematical benchmarks, e.g., GSM8K, it remains unclear whether small to medium-sized open LLMs can achieve similar performance, questioning their reliability. To close this gap, we propose a post-training approach leveraging a mixture of opinions (MoO) from weaker ancillary LLMs to enhance a (relatively) stronger LLM's reasoning. For that, each post-training sample is augmented with Chain-of-Thought (CoT) reasoning steps and answers from ancillary LLMs, enabling the main LLM to learn from diverse perspectives. We compare MoO with standard supervised fine-tuning (SFT), few-shot prompting, and the Mixture of Agents (MoA) method on mathematical reasoning benchmarks. Our results show that incorporating weaker LLMs' opinions improves mathematical reasoning by an average of 5%, highlighting the value of diverse perspectives in reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。