让大模型自己生成解题思路，比传统方法平均提升7%。
Auto-Evolve: Enhancing Large Language Model's Performance via Self-Reasoning Framework
- 模型自动生成动态解题步骤，无需预设模板。
- 在BigBench-Hard上平均比CoT高7%，最高提升10.4%。
- 适合想提升模型推理能力的研究者和开发者。
近期的提示工程策略，如思维链（Chain-of-Thought, CoT）和自发现（Self-Discover），已证明能显著提升大语言模型（LLM）的推理能力。然而，这些最先进的提示方法依赖于固定或静态的初始推理模块，如“逐步思考”或“分解问题”，难以灵活应对多样任务。本文提出Auto-Evolve框架，使大模型能够自动生成动态推理模块与下游行动方案，在Claude 2.0、Claude 3 Sonnet、Mistral Large和GPT 4上评估，均显著优于现有SOTA方法。在BigBench-Hard数据集上，该框架平均性能较CoT提升7%，最高达10.4%。其核心创新包括：a) 动态生成适配任务的推理模块，契合人类推理范式，无需预设模板；b) 引入迭代优化机制，逐轮改进指令引导，相较单步操作平均提升2.8%。
原文摘要 · Abstract (English)
Recent advancements in prompt engineering strategies, such as Chain-of-Thought (CoT) and Self-Discover, have demonstrated significant potential in improving the reasoning abilities of Large Language Models (LLMs). However, these state-of-the-art (SOTA) prompting strategies rely on single or fixed set of static seed reasoning modules like "think step by step" or "break down this problem" intended to simulate human approach to problem-solving. This constraint limits the flexibility of models in tackling diverse problems effectively. In this paper, we introduce Auto-Evolve, a novel framework that enables LLMs to self-create dynamic reasoning modules and downstream action plan, resulting in significant improvements over current SOTA methods. We evaluate Auto-Evolve on the challenging BigBench-Hard (BBH) dataset with Claude 2.0, Claude 3 Sonnet, Mistral Large, and GPT 4, where it consistently outperforms the SOTA prompt strategies. Auto-Evolve outperforms CoT by up to 10.4% and on an average by 7% across these four models. Our framework introduces two innovations: a) Auto-Evolve dynamically generates reasoning modules for each task while aligning with human reasoning paradigm, thus eliminating the need for predefined templates. b) We introduce an iterative refinement component, that incrementally refines instruction guidance for LLMs and helps boost performance by average 2.8% compared to doing it in a single step.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。