arXiv:2604.14531cs.AI2026-04

用大模型自身日志训练轻量代理,自动决定何时接管推理任务。

TRACER: Trace-Based Adaptive Cost-Efficient Routing for LLM Classification

  • 基于生产日志训练轻量代理模型,动态判断是否可替代主模型
  • 在77类意图识别中实现83%-100%代理覆盖,150类任务完全替代
  • 提供可解释性分析,明确代理能处理和不能处理的输入区域

每次调用大模型分类接口都会生成一个已记录在生产日志中的带标签输入输出对。这些配对构成了免费且持续增长的训练集:基于它们训练的轻量级代理模型可吸收大部分未来流量,边际推理成本接近零。核心问题是何时代理足够可靠、它能处理哪些任务、又将哪些任务回退给原模型,以及这一边界如何随数据积累演化。我们提出TRACER(Trace-based Adaptive Cost-Efficient Routing),一个开源系统,利用大模型自身的生产轨迹训练机器学习代理,并通过一致性门控机制控制部署:仅当代理与大模型的一致性超过用户设定阈值α时才启用。为使路由边界透明,TRACER生成可解释性可视化结果,描述代理能处理的输入区域、其性能饱和点及回退原因。在77类意图识别基准上,使用Sonnet 4.6作为教师模型,TRACER实现了83%-100%的代理覆盖;在150类任务中,代理完全取代教师模型。在自然语言推理任务中,一致性门控正确拒绝部署,因嵌入表示无法支持可靠分类。系统已开源。

原文摘要 · Abstract (English)

Every call to an LLM classification endpoint produces a labeled input-output pair already retained in production logs. These pairs constitute a free, growing training set: a lightweight surrogate trained on them can absorb a significant portion of future traffic at near-zero marginal inference cost. The open questions are when the surrogate is reliable enough to deploy, what it handles versus defers, and how that boundary evolves as data accumulates. We introduce TRACER (Trace-based Adaptive Cost-Efficient Routing), an open-source system that trains ML surrogates on an LLM's own production traces and governs deployment through a parity gate: the surrogate is activated only when its agreement with the LLM exceeds a user-specified threshold α. To make the routing boundary transparent, TRACER generates interpretability artifacts describing which input regions the surrogate handles, where it plateaus, and why it defers. On a 77-class intent benchmark with a Sonnet 4.6 teacher, TRACER achieves 83-100% surrogate coverage depending on the quality target α; on a 150-class benchmark, the surrogate fully replaces the teacher. On a natural language inference task, the parity gate correctly refuses deployment because the embedding representation cannot support reliable separation. The system is available as open-source software.

大模型优化智能路由代理模型可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。