通过预测纠错与错误替换，实现更高效的LLM推理路由。
Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference
- 分别预测小模型正确时大模型能否纠错、是否会错误替换
- 在多个数据集上相比基线提升12%~18%的效率-精度平衡
- 仅需小模型输出统计，适合资源受限场景部署
大型语言模型（LLM）级联通过小模型处理简单请求，将部分请求升级到大模型。现有路由器主要依据小模型的不确定性进行决策，但忽略了关键事实：升级只有在大模型能纠正小模型错误时才有价值，若大模型反而将正确答案改为错误，则有害。本文提出签名救援路由（Signed Rescue Routing, SRR），一种带预算的路由方法，分别预测这两种事件并按其差值排序。我们证明该符号条件增益是在固定升级预算下的贝叶斯最优路由评分。SRR仅需部署时的小模型输出统计，并引入轻量级双头路由器。在Qwen3-4B和Qwen3-8B上，针对MMLU、HellaSwag和ARC-Challenge中的TBD个示例进行评估，跨准确率-计算曲线，SRR达到面积为TBD，优于基于学习的小模型误差预测器（TBD）和熵路由（TBD）。结果表明，预测增量价值而非模型不确定性，是高效LLM级联的简洁有效目标。
原文摘要 · Abstract (English)
Large language model (LLM) cascades answer easy requests with a small model and escalate selected requests to a larger model. Most routers prioritize examples on which the small model appears uncertain or likely to be wrong. This proxy ignores a decisive fact: escalation is useful only when the large model corrects the small model, and it is harmful when the large model replaces a correct answer with an incorrect one. We introduce Signed Rescue Routing (SRR), a budgeted routing method that predicts these two events separately and ranks requests by their difference. We show that this signed conditional gain is the Bayes-optimal routing score under a fixed escalation budget. SRR requires only the small model's output statistics at deployment and adds a lightweight two-head router. We evaluate SRR with Qwen3-4B and Qwen3-8B on TBD examples from MMLU, HellaSwag, and ARC-Challenge. Across the accuracy-compute curve, SRR reaches an area of TBD, compared with TBD for a learned small-model error predictor and TBD for entropy routing. These results show that predicting incremental value, rather than model uncertainty, is a simple and effective objective for efficient LLM cascades.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。