arXiv:2509.26626cs.LG2025-09被引 19

通过递归聚合提升大模型推理能力,显著增强复杂问题解决效果。

Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models

  • 采用递归聚合策略,融合多轮自修正与并行解法优势。
  • 在多项基准测试中超越主流模型,如Gemini 3 Flash接近顶尖水平。
  • 适合需要深度推理的场景,尤其对小模型性能提升明显。

推理时扩展方法通过增加推理阶段的计算量来提升大语言模型的能力。推理计算可通过并行选择多个独立解法或串行自修正来扩展。本文提出递归自聚合(RSA),受进化方法启发,结合了并行与串行扩展的优点。每轮RSA通过聚合候选推理链子集,生成更优解集,作为下一轮的候选池。实验表明,RSA在不同任务、模型族和规模下均随计算预算增加显著提升性能。特别地,使用Gemini 3 Flash的RSA接近ARC-AGI-2公开排行榜榜首。RSA还使Qwen3-4B-Instruct-2507在AIME-25、HMMT-25、Reasoning Gym、LiveCodeBench-v6和SuperGPQA等任务上表现媲美更大模型(如DeepSeek-R1和o3-mini),优于纯并行与串行扩展策略。此外,我们提出一种新的聚合感知强化学习方法,训练模型更有效地组合解法,带来显著性能提升。

原文摘要 · Abstract (English)

Test-time scaling methods improve the capabilities of large language models (LLMs) by increasing the amount of compute used during inference to make a prediction. Inference-time compute can be scaled in parallel by choosing among multiple independent solutions or sequentially through self-refinement. We propose Recursive Self-Aggregation (RSA), a test-time scaling method inspired by evolutionary methods that combines the benefits of both parallel and sequential scaling. Each step of RSA refines a population of candidate reasoning chains through aggregation of subsets to yield a population of improved solutions, which are then used as the candidate pool for the next iteration. Empirically, RSA delivers substantial performance gains with increasing compute budgets across diverse tasks, model families and sizes. Notably, RSA with Gemini 3 Flash attains performance near the top of the ARC-AGI-2 public leaderboard. RSA also enables Qwen3-4B-Instruct-2507 to achieve competitive performance with larger reasoning models, including DeepSeek-R1 and o3-mini (high), outperforming purely parallel and sequential scaling strategies across AIME-25, HMMT-25, Reasoning Gym, LiveCodeBench-v6, and SuperGPQA. We further propose a novel aggregation-aware reinforcement learning approach that yields significant performance gains by training the model to combine solutions.

大模型推理自修正测试时扩展聚合策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。