让智能体按预算动态选模型,省钱又高效。
Budget-Aware Agentic Routing via Boundary-Guided Training
- 用边界策略训练,自动判断任务难易并指导选型
- 在相同成功率下,成本比基线低得多,且能严格控预算
- 适合需要长期任务和严格开支控制的智能体系统
随着大语言模型演变为执行长程任务的自主智能体,每一步都调用高性能模型已不可持续。传统模型路由适用于单轮查询,但智能体路由是序列依赖问题:早期错误会累积,反馈常在任务末尾,且部署时常有严格的单任务预算限制。本文提出预算感知的智能体路由方法,每步在低成本与高成本模型间决策,以优化成本-成功权衡,并满足严格预算约束。通过边界引导训练(Boundary-Guided Training),利用始终使用小模型与始终使用大模型两种边界策略构建难度分类体系,并在稀疏奖励下锚定学习。方法先通过分层采样生成成本高效的轨迹用于监督微调,再采用边界相对奖励与参考引导优势的策略优化(BoPO),防止出现仅选廉价模型却失败的退化解。实验表明,该方法显著提升效率前沿,在保持强基线性能的同时大幅降低消耗,并具备良好的推理时预算约束泛化能力。整体上,本工作建立了智能体路由的基础框架,推动从静态选模转向动态、预算感知的序列决策范式。
原文摘要 · Abstract (English)
As large language models (LLMs) evolve into autonomous agents that execute long-horizon workflows, invoking a high-capability model at every step becomes economically unsustainable. While model routing is effective for single-turn queries, agentic routing is a sequential, path-dependent problem: early mistakes compound, feedback is often at the end of the episode, and deployments often demand strict per-task spending limits. We propose Budget-Aware Agentic Routing, which selects between a cheap and an expensive model at each step to optimize the cost--success frontier and to operate under strict per-task budgets. We propose Boundary-Guided Training, which leverages two boundary policies (always-small vs.\ always-large) to build a difficulty taxonomy and to anchor learning under sparse rewards. Our approach warms start with boundary-guided SFT data synthesis via stratified sampling of cost-efficient trajectories, then applies Boundary-Guided Policy Optimization (BoPO), combining boundary-relative rewards with a reference-guided advantage to avoid degenerate cheap-failure solutions. Experiment results show that our method improves the efficiency frontier, matching strong routing baselines at substantially lower cost while demonstrating generalization to strict inference-time budget constraints. Overall, our work establishes a foundational framework for agentic routing, shifting the paradigm from static model selection to dynamic, budget-aware sequential decision-making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。