让大模型分头协作，提升复杂问题推理能力
Cooperative Strategic Planning Enhances Reasoning Capabilities in Large Language Models
- 拆解推理步骤，由规划与推理双代理协同完成
- 在LogiQA和BBH上分别提升9.94%和3.09%准确率
- 适合需要多步逻辑推理的AI系统开发人员
提升大语言模型（LLM）的推理能力对于解决复杂多步问题至关重要。虽然多智能体框架在增强推理能力方面展现出潜力，但智能体间缺乏有效协作仍限制其表现，尤其在多步推理任务中。本文提出一种新型协作式多智能体推理框架CoPlanner，通过分离推理步骤并为不同智能体分配特定职责实现优化。CoPlanner包含两个基于LLaMA-3-8B的智能体：规划代理提供高层策略提示，推理代理根据提示推导答案。通过近端策略优化（PPO）在交互推理过程中训练规划代理的策略，CoPlanner在LogiQA上较此前最优方法提升9.94%，在BBH上提升3.09%。结果表明，规划代理的引导作用及智能体间的有效协作显著提升了多步推理性能。
原文摘要 · Abstract (English)
Enhancing the reasoning capabilities of large language models (LLMs) is crucial for enabling them to tackle complex, multi-step problems. Multi-agent frameworks have shown great potential in enhancing LLMs' reasoning capabilities. However, the lack of effective cooperation between LLM agents hinders their performance, especially for multi-step reasoning tasks. This paper proposes a novel cooperative multi-agent reasoning framework (CoPlanner) by separating reasoning steps and assigning distinct duties to different agents. CoPlanner consists of two LLM agents: a planning agent and a reasoning agent. The planning agent provides high-level strategic hints, while the reasoning agent follows these hints and infers answers. By training the planning agent's policy through the interactive reasoning process via Proximal Policy Optimization (PPO), the LLaMA-3-8B-based CoPlanner outperforms the previous best method by 9.94\% on LogiQA and 3.09\% on BBH. Our results demonstrate that the guidance from the planning agent and the effective cooperation between the agents contribute to the superior performance of CoPlanner in tackling multi-step reasoning problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。