arXiv:2605.08083cs.CL2026-05被引 1

用自动探索框架让大模型自己发现推理优化策略,省时省力还更高效。

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

论文配图:LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling
图 1 · 摘自论文原文
  • 构建可自动搜索的推理环境,让模型自主设计优化策略。
  • 在数学推理任务中,新策略在准确率与计算成本间表现更优。
  • 发现的策略能跨数据集和模型规模通用,且仅需39.9美元和160分钟。

测试时扩展(TTS)通过推理阶段增加计算量来提升大语言模型性能,但现有策略多为人工设计,依赖直觉调参,未充分探索计算分配空间。本文提出环境驱动的AutoTTS框架,将设计焦点从单个启发式规则转向可自动发现策略的环境。核心在于构造可控、反馈廉价且频繁的发现环境:以预收集的推理轨迹与探测信号为基础,将宽度-深度TTS建模为控制器合成问题,控制器决定分支、继续、探测、剪枝或终止,且无需重复调用大模型即可快速评估。引入beta参数化使搜索空间更精细可控,并通过细粒度执行轨迹反馈提升代理诊断能力。实验表明,所发现策略在数学推理基准上优于强手动基线,在准确率-成本权衡上表现更佳;策略具备跨数据集与模型规模的泛化能力,整个发现过程仅耗时160分钟、花费39.9美元。代码与数据开源于https://github.com/zhengkid/AutoTTS。

原文摘要 · Abstract (English)

Test-time scaling (TTS) has become an effective approach for improving large language model performance by allocating additional computation during inference. However, existing TTS strategies are largely hand-crafted: researchers manually design reasoning patterns and tune heuristics by intuition, leaving much of the computation-allocation space unexplored. We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTS heuristics to environments where TTS strategies can be discovered automatically. The key to AutoTTS lies in environment construction: the discovery environment must make the control space tractable and provide cheap, frequent feedback for TTS search. As a concrete instantiation, we formulate width--depth TTS as controller synthesis over pre-collected reasoning trajectories and probe signals, where controllers decide when to branch, continue, probe, prune, or stop and can be evaluated cheaply without repeated LLM calls. We further introduce beta parameterization to make the search tractable and fine-grained execution trace feedback to improve discovery efficiency by helping the agent diagnose why a TTS program fails. Experiments on mathematical reasoning benchmarks show that the discovered strategies improve the overall accuracy--cost tradeoff over strong manually designed baselines. The discovered strategies generalize to held-out benchmarks and model scales, while the entire discovery costs only $39.9 and 160 minutes. Our data, and code will be open-source at https://github.com/zhengkid/AutoTTS.

测试时扩展自动化设计推理优化智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。