让大模型自动选最适合的推理格式，提升推理准确性
Format-Adapter: Improving Reasoning Capability of LLMs by Adapting Suitable Format
- 用大模型自动生成并筛选推理格式，替代人工标注
- 在数学和常识推理任务上平均提升4.3%性能
- 适合需要高可靠推理的场景，如考试答题系统
生成多个答案并进行投票是缓解大语言模型推理不一致的有效方法。先前研究发现，使用多种推理格式比单一格式更有效。然而，现有方法依赖人工标注的格式，可能不适用于所有任务，且标注成本高。为解决此问题，我们提出通过生成和选择合适格式来适应具体任务。首先，我们定义了生成多答案时的推理误差度量方法；随后，提出Format-Adapter，利用大模型生成并选择最小化该误差的推理格式。我们在数学和常识推理任务上进行实验，结果表明，Format-Adapter相比之前方法平均提升4.3%，验证了其有效性。
原文摘要 · Abstract (English)
Generating and voting multiple answers is an effective method to mitigate reasoning inconsistencies of large language models (LLMs). Prior works have shown that multiple reasoning formats outperform a single format when generating multiple answers. However, previous works using multiple formats rely on formats labeled by humans, which could be unsuitable for all tasks and have high labeling costs. To address this issue, we adapt suitable formats to the given tasks by generating and selecting formats. We first propose how to measure the reasoning error when generating multiple answers. Then, we introduce Format-Adapter, which utilizes LLMs to generate and select suitable reasoning formats by minimizing the error measurement we present. We conduct experiments on math and commonsense reasoning tasks, where Format-Adapter achieves a 4.3% performance improvement on average over previous works, demonstrating the effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。