arXiv:2605.14241cs.LG2026-05被引 2

通过质量-延迟匹配优化大模型工具路由,提升响应准确率

Latency-Quality Routing for Functionally Equivalent Tools in LLM Agents

论文配图:Latency-Quality Routing for Functionally Equivalent Tools in LLM Agents
图 1 · 摘自论文原文
  • 基于上下文强化学习,按单位服务周期质量排序工具
  • 在搜索和检索任务中提升准确率2.18至18个百分点
  • 适合高负载、多供应商差异大的智能代理系统

增强型大语言模型代理越来越多地通过多个功能等价的提供者访问相同类型的工具,如网络搜索API、检索器或共享接口背后的LLM后端。这带来了运行时负载下的提供商路由问题:路由器需在延迟、可靠性和答案质量各异的提供者间做出选择,且部署时通常无真实标签。我们提出LQM-ContextRoute,一种面向同功能工具提供者的上下文老虎机路由算法。其核心设计为质量-延迟匹配:不将低延迟视为对差答案的加性补偿,而是按每服务周期预期答案质量进行排序。该方法结合容量感知评分、查询特定质量估计及大模型作为评判者反馈,可在线适应负载变化与提供者质量差异。在主流网络搜索负载基准上,相较于SW-UCB,LQM-ContextRoute提升F1达+2.18个百分点,同时保持在延迟-质量前沿;在高异质性StrategyQA场景中,避免了加性奖励崩溃,准确率最高提升+18个百分点;在异质检索器池中,提升NDCG达+2.91至+3.22个百分点。结果表明,在运行压力与提供者质量异质性共存时,将延迟视为服务能力的路由策略更具优势。

原文摘要 · Abstract (English)

Tool-augmented LLM agents increasingly access the same tool type through multiple functionally equivalent providers, such as web-search APIs, retrievers, or LLM backends exposed behind a shared interface. This creates a provider-routing problem under runtime load: the router must choose among providers that differ in latency, reliability, and answer quality, often without gold labels at deployment time. We introduce LQM-ContextRoute, a contextual bandit router for same-function tool providers. Its key design is latency-quality matching: instead of letting low latency offset poor answers in an additive reward, the router ranks providers by expected answer quality per service cycle. It combines this capacity-aware score with query-specific quality estimation and LLM-as-judge feedback, allowing it to adapt online to both load changes and provider-quality differences. On the main web-search load benchmark, LQM-ContextRoute improves F1 by +2.18 pp over SW-UCB while staying on the latency-quality frontier. In a high-heterogeneity StrategyQA setting, LQM-ContextRoute avoids additive-reward collapse and improves accuracy by up to +18 pp over SW-UCB; on heterogeneous retriever pools, it improves NDCG by +2.91--+3.22 pp over SW-UCB. These results show that same-function tool routing benefits from treating latency as service capacity, especially when runtime pressure and provider-quality heterogeneity coexist.

工具路由大模型代理延迟优化强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。