通过反思与拆解问题，让大模型更聪明地解题。
Enhancing LLM Problem Solving with REAP: Reflection, Explicit Problem Deconstruction, and Advanced Prompting
- 引导模型反思问题并拆解成小任务，再生成辅助上下文。
- GPT-4o-mini性能提升112.93%，成本仅为o1-preview的1%。
- 输出更清晰易懂，适合需要可解释性的实际应用。
大型语言模型（LLMs）已改变自然语言处理领域，但提升其在复杂推理任务中的问题解决能力仍是长期挑战。本文提出REAP（反思、显式问题拆解与高级提示）方法，属于动态上下文生成框架的创新方案。REAP通过引导模型反思查询、将问题拆解为可管理组件，并生成相关上下文来增强求解过程。我们在一个专门设计用于暴露LLM局限性的数据集上评估REAP，对比了零样本提示与REAP增强提示在六种先进模型上的表现：OpenAI的o1-preview、o1-mini、GPT-4o、GPT-4o-mini，Google的Gemini 1.5 Pro，以及Claude 3.5 Sonnet。结果表明，性能显著提升：o1-mini提升40.97%，GPT-4o提升66.26%，GPT-4o-mini提升112.93%。尽管o1-preview基线表现已很强，仍观察到小幅增益。此外，REAP具成本效益：例如，价格约为o1-preview 1/100的GPT-4o-mini达到了竞争力的结果。该方法还提升了输出清晰度，使人类更易理解推理过程，便于发现和修复问题。这些成果证明，REAP能显著增强多种应用场景下大模型的能力，兼具性能与效率优势。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have transformed natural language processing, yet improving their problem-solving capabilities, particularly for complex, reasoning-intensive tasks, remains a persistent challenge. This paper introduces the REAP (Reflection, Explicit Problem Deconstruction, and Advanced Prompting) method, an innovative approach within the dynamic context generation framework. REAP guides LLMs through reflection on the query, deconstructing it into manageable components, and generating relevant context to enhance the solution process. We evaluated REAP using a dataset designed to expose LLM limitations, comparing zero-shot prompting with REAP-enhanced prompts across six state-of-the-art models: OpenAI's o1-preview, o1-mini, GPT-4o, GPT-4o-mini, Google's Gemini 1.5 Pro, and Claude 3.5 Sonnet. The results demonstrate notable performance gains, with o1-mini improving by 40.97%, GPT-4o by 66.26%, and GPT-4o-mini by 112.93%. Despite the already strong baseline performance of OpenAI's o1-preview, modest gains were observed. Beyond performance improvements, REAP offers a cost-effective solution; for example, GPT-4o-mini, which is approximately 100 times cheaper than o1-preview, delivered competitive results. REAP also improves the clarity of model outputs, making it easier for humans to understand the reasoning behind the results and simplifying the process of identifying and addressing any issues. These findings demonstrate REAP's potential to greatly improve the capabilities of LLMs, providing both better performance and increased cost-efficiency across a wide range of applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。