为智能体应用设计任务级路由,提升长流程任务的准确率与延迟平衡。
TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI

- 任务入口时选定模型并固定后续调用,避免逐次决策偏差。
- 利用任务完成后的最终奖励优化策略,实现准确率与延迟双提升。
- 无需预估任务复杂度,适合企业级长周期智能体应用部署。
选择具备不同成本-质量权衡的大型语言模型(LLMs)进行路由,已成为企业AI部署的核心功能。现有路由器对每个LLM调用独立决策,但智能体应用以长周期工作流执行,其质量取决于延迟的任务级结果。这种不匹配导致逐次路由器无法正确将反馈归因于单个路由决策。为此,我们提出TRACE-Router,一种以任务为单位的路由框架,使路由与监督单元对齐。TRACE-Router在任务进入时通过上下文老虎机算法选定模型,并固定后续所有LLM调用的后端,使用任务的最终奖励更新策略,同时兼顾准确率与延迟。通过利用延迟的任务反馈,TRACE-Router学习适应性路由策略,无需显式估计任务复杂度。在三个智能体基准测试中,该方法持续改善准确率-延迟权衡,达到非支配的帕累托前沿点。在tau2-Bench上,其准确率比延迟匹配的模型插值高7-8个百分点;在Terminal-Bench上,准确率比最强单模型基线高7.1个百分点,且延迟降低36%。
原文摘要 · Abstract (English)
Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI. Existing routers, primarily make independent routing decisions for each LLM call. However, agentic applications execute as long-horizon workflows whose quality is determined only by a delayed, task-level outcome. This mismatch prevents per-call routers from correctly attributing feedback to individual routing decisions. Towards mitigating this, we present TRACE-Router, a task-level routing framework that aligns routing with the unit of supervision. TRACE-Router assigns each task to a model once at admission using a contextual bandit, pins all subsequent LLM calls to the selected backend, and updates its policy using the task's terminal reward, jointly accounting for accuracy and latency. By leveraging delayed task feedback, TRACE-Router learns routing policies that adapt to the workload while avoiding explicit task-complexity estimation. Across three agentic benchmarks, TRACE-Router consistently improves the accuracy-latency trade-off, achieving non-dominated Pareto frontier points. On tau2-Bench, it outperforms latency-matched interpolation between individual models by 7-8 accuracy points, while on Terminal-Bench it achieves 7.1 higher accuracy points than the strongest single model baseline with 36% lower latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。