arXiv:2602.11170cs.CL2026-02

用多智能体协作提升大模型的算法推理能力,准确率最高提升250%。

PRIME: Policy-Reinforced Iterative Multi-agent Execution for Algorithmic Reasoning in Large Language Models

  • 设计三类智能体协同推理,通过迭代验证防止错误累积。
  • 在86个任务上平均准确率达93.8%,远超基线26.8%。
  • 适合需要长期状态追踪的复杂算法任务,小模型效果接近大模型。

大语言模型在多种推理任务中表现优异,但在算法推理方面仍存局限。为此,我们提出PRIME(Policy-Reinforced Iterative Multi-agent Execution)框架,包含执行器、验证器和协调器三类专用智能体,通过群体相对策略优化进行训练。为全面评估,我们构建了目前最大的算法推理基准PRIME-Bench,涵盖12类共86个任务,包含51,600个实例,任务覆盖排序算法、图与树结构、自动机与状态机、符号推理及约束谜题,执行轨迹超过一百万步。相比基线方法,PRIME将平均准确率从26.8%提升至93.8%,相对提升250%。在需持续状态追踪的任务上提升尤为显著:图灵机模拟准确率从9%升至92%,长除法从16%升至94%。消融实验表明,迭代验证是主要贡献因素,有效阻止错误传播。跨模型规模分析(8B-120B参数)显示,小模型获益更明显,性能可媲美8倍大的模型。

原文摘要 · Abstract (English)

Large language models have demonstrated remarkable capabilities across diverse reasoning tasks, yet their performance on algorithmic reasoning remains limited. To handle this limitation, we propose PRIME (Policy-Reinforced Iterative Multi-agent Execution), a framework comprising three specialized agents, an executor for step-by-step reasoning, a verifier for constraint checking, and a coordinator for backtracking control, optimized through group relative policy optimization. For comprehensive evaluation, we introduce PRIME-Bench, the largest algorithmic reasoning benchmark to date, comprising 86 tasks across 12 categories with 51,600 instances. Tasks span sorting algorithms, graph and tree structures, automata and state machines, symbolic reasoning, and constraint-based puzzles, with execution traces reaching over one million steps. Compared to baseline approach, PRIME improves average accuracy from 26.8% to 93.8%, a 250% relative gain. The largest improvements occur on tasks requiring sustained state tracking, with Turing machine simulation improving from 9% to 92% and long division from 16% to 94%. Ablation studies identify iterative verification as the primary contributor, preventing the error propagation that causes baseline approaches to fail catastrophically. Analysis across model scales (8B-120B parameters) reveals that smaller models benefit disproportionately, achieving accuracy comparable to models 8x larger.

算法推理多智能体大模型验证机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。