用思维链合成答案，让模型在错误选项中也能找到正确解。
CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis
- 通过分析多个错误答案的互补信息，用思维链生成更优解。
- 在MATH数据集上，Llama3-8B提升11.8%，GPT-4o提升10.3%。
- 小模型可训练提升大模型推理准确率，适合资源有限的研究者。
当前的推理扩展方法如Self-consistency和Best-of-N在复杂推理任务中有效提升了大语言模型（LLM）的准确性，但严重依赖候选答案质量，当所有候选答案均错误时无法生成正确结果。本文提出一种新型推理扩展策略——基于思维链的合成器（CoT-based Synthesizer），利用思维链推理分析多个候选答案中的互补信息，即使所有候选答案均有误，仍能合成更优答案。为实现轻量且低成本的部署，我们设计了自动化数据生成管道，生成多样化训练数据。由此训练的小型模型可显著提升大型模型（包括API调用模型）的推理准确率。在四个基准数据集上，七种策略模型的实验表明，本方法性能显著提升：在MATH数据集上，Llama3-8B提升11.8%，GPT-4o提升10.3%。相关训练数据与代码已公开于https://github.com/RUCKBReasoning/CoT-based-Synthesizer。
原文摘要 · Abstract (English)
Current inference scaling methods, such as Self-consistency and Best-of-N, have proven effective in improving the accuracy of LLMs on complex reasoning tasks. However, these methods rely heavily on the quality of candidate responses and are unable to produce correct answers when all candidates are incorrect. In this paper, we propose a novel inference scaling strategy, CoT-based Synthesizer, which leverages CoT reasoning to synthesize superior answers by analyzing complementary information from multiple candidate responses, even when all candidate responses are flawed. To enable a lightweight and cost-effective implementation, we introduce an automated data generation pipeline that creates diverse training data. This allows smaller LLMs trained on this data to improve the inference accuracy of larger models, including API-based LLMs. Experimental results across four benchmark datasets with seven policy models demonstrate that our method significantly enhances performance, with gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on the MATH dataset. The corresponding training data and code are publicly available on https://github.com/RUCKBReasoning/CoT-based-Synthesizer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。