让大模型通过迭代优化实现更优推理,大幅降低参数需求。
MARINE: Theoretical Optimization and Design for Multi-Agent Recursive IN-context Enhancement
- 用递归迭代方式持续优化推理轨迹,突破单次输出限制。
- 80B模型用MARINE后达到1000B模型性能,参数减少超90%。
- 在相同算力下,生成质量优于传统采样排名方法,适合高效对齐优化。
基于大语言模型的智能体虽具强大推理能力,但实际应用常受限于单次输出,未能发挥全部潜力。本文提出MARINE(多智能体递归上下文增强)框架,将测试时推理重新定义为对持续参考轨迹的迭代优化,突破传统一次或多次采样范式。理论分析表明,在固定调用预算下,最小可行批大小可最大化预期性能提升,而对数增长的批处理策略可实现无约束持续改进。在BrowserComp-ZH基准上的全面评估显示,685B参数实现46.0% pass@1准确率,达到当前最优。更重要的是,80B参数模型经MARINE增强后,性能媲美独立1000B参数智能体,参数需求降低超一个数量级。在固定计算预算下,其生成样本质量优于传统采样-排序策略,显著提升后训练效率。
原文摘要 · Abstract (English)
Large Language Model (LLM)-based agents demonstrate advanced reasoning capabilities, yet practical constraints frequently limit outputs to single responses, leaving significant performance potential unrealized. This paper introduces MARINE (Multi-Agent Recursive IN-context Enhancement), a theoretically grounded framework that reconceptualizes test-time reasoning as iterative refinement of a persistent reference trajectory, fundamentally departing from conventional one-shot or multi-sample paradigms. The MARINE refinement operator systematically converts a base model's pass@N capabilities into near-optimal pass@1 performance. Rigorous theoretical analysis establishes that minimal feasible batches maximize expected performance gains under fixed invocation budgets, while logarithmically growing batch schedules ensure continuous improvement without computational constraints. Comprehensive evaluation on the BrowserComp-ZH benchmark demonstrates state-of-the-art results, with a 685B-parameter implementation achieving 46.0% pass@1 accuracy. Meanwhile, MARINE establishes a new paradigm for parameter-efficient reasoning: an 80B-parameter model augmented with MARINE matches the performance of standalone 1000B-parameter agents, reducing parameter requirements by over an order of magnitude. Notably, within a fixed computational budget, the proposed MARINE delivers higher-quality samples to alignment and optimization processes than traditional sampling-and-ranking strategies. Consequently, it has great potential to boost post-training efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。