arXiv:2604.09741cs.LGcs.AI2026-04中稿 · ICLR

用引导模型提升大模型推理效率,降低调用成本。

ExecTune: Effective Steering of Black-Box LLMs with Guide Models

  • 设计引导-核心策略框架,通过引导模型生成可执行的推理步骤
  • 在数学和代码任务上准确率最高提升9.2%,推理成本降低22.4%
  • 支持模块化更新引导模型,无需重训核心模型

针对通过黑盒API部署的大语言模型,持续推理成本常高于一次性训练成本。为此,我们研究一类称为引导-核心策略(GCoP)的系统,其中引导模型生成结构化策略,由黑盒核心模型执行。该框架涵盖基础、监督及顾问式方法,主要区别在于引导模型的训练方式。我们基于成本敏感效用目标形式化了GCoP,并发现最终性能由引导平均可执行性决定——即引导生成的策略被核心模型准确执行的概率。分析表明,现有实现常因未优化部署约束下的可执行性,导致策略脆弱且计算低效。为此,我们提出ExecTune,一种结合教师引导采样、监督微调与结构感知强化学习的系统性训练方法,直接优化语法正确性、执行成功率与成本效率。在数学推理与代码生成基准测试中,使用ExecTune的GCoP相较最先进基线准确率最高提升9.2%,推理成本降低22.4%。它使Claude Haiku 3.5在数学与代码任务上超越Sonnet 3.5,且在38%更低成本下,仅比Sonnet 4低1.7%绝对准确率。此外,该框架支持通过更新引导模型实现模块化适配,无需重训核心模型。

原文摘要 · Abstract (English)

For large language models deployed through black-box APIs, recurring inference costs often exceed one-time training costs. This motivates composed agentic systems that amortize expensive reasoning into reusable intermediate representations. We study a broad class of such systems, termed Guide-Core Policies (GCoP), in which a guide model generates a structured strategy that is executed by a black-box core model. This abstraction subsumes base, supervised, and advisor-style approaches, which differ primarily in how the guide is trained. We formalize GCoP under a cost-sensitive utility objective and show that end-to-end performance is governed by guide-averaged executability: the probability that a strategy generated by the guide can be faithfully executed by the core. Our analysis shows that existing GCoP instantiations often fail to optimize executability under deployment constraints, resulting in brittle strategies and inefficient computation. Motivated by these insights, we propose ExecTune, a principled training recipe that combines teacher-guided acceptance sampling, supervised fine-tuning, and structure-aware reinforcement learning to directly optimize syntactic validity, execution success, and cost efficiency. Across mathematical reasoning and code-generation benchmarks, GCoP with ExecTune improves accuracy by up to 9.2% over prior state-of-the-art baselines while reducing inference cost by up to 22.4%. It enables Claude Haiku 3.5 to outperform Sonnet 3.5 on both math and code tasks, and to come within 1.7% absolute accuracy of Sonnet 4 at 38% lower cost. Beyond efficiency, GCoP also supports modular adaptation by updating the guide without retraining the core.

大模型优化推理效率黑盒模型引导策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。