arXiv:2605.06350cs.LGcs.AI2026-05

用决策理论分析大模型级联的性价比,发现结构成本是瓶颈。

Is Escalation Worth It? A Decision-Theoretic Characterization of LLM Cascades

  • 基于约束优化与对偶理论,建立两级模型级联的成本-质量边界模型。
  • 多模型级联的最优性能是所有两两组合的包络线,切换点由最优对决定。
  • 轻量预生成路由优于级联策略,因避免了廉价模型的无效计算开销。

模型级联通过让低成本大模型在置信度低时转接高成本模型,来权衡部署阶段的成本与质量。现有方法大多将转接阈值当作经验超参数,缺乏对模型池中成本-质量边界的几何性质指导。本文基于约束优化与对偶理论,构建决策论框架:对于双模型级联,证明其成本-质量边界在置信度支持的递减收益区域具有分段凹性,预算与质量约束间由互为影子价格关联。对于 $k$ 模型池,确定由确定性两模型阈值级联可实现的边界为 $inom{k}{2}$ 个两两级联的逐点包络,切换点对应最优组合变化。对于 $k$-模型级联,导出一阶条件,即单一影子价格使各阶段间边际质量/成本相等。在五项基准(MATH、MMLU、TriviaQA、SimpleQA、LiveCodeBench)上验证,覆盖八款来自五家厂商的模型。结果表明,在确定性阈值级联类中,全固定链路表现低于两两组合包络;优化子序列级联未带来实际提升。轻量预生成路由器在四项数据集上超越最佳级联策略,主因在于避免廉价模型在直接发送至大模型时的生成开销,而非更强的路由信号。结果表明,级联性能受限于结构性成本——因级联必须先支付廉价模型代价,才触发升级决策,而非中间阶段不足所致。

原文摘要 · Abstract (English)

Model cascades, in which a cheap LLM defers to an expensive one on low-confidence queries, are widely used to navigate the cost-quality tradeoff at deployment. Existing approaches largely treat the deferral threshold as an empirical hyperparameter, with limited guidance on the geometry of the resulting cost-quality frontier over a model pool. We develop a decision-theoretic framework grounded in constrained optimization and duality. For a two-model cascade, we establish piecewise concavity of the cost-quality frontier on decreasing-benefit regions of the confidence support, with reciprocal shadow prices linking the budget- and quality-constrained formulations. Given a pool of $k$ models, we characterize the frontier achievable by deterministic two-model threshold cascades as the pointwise envelope over $\binom{k}{2}$ pairwise cascades, with switching points where the optimal pair changes. For $k$-model cascades, we derive first-order conditions in which a single shadow price equalizes marginal quality-per-cost across stage boundaries. We validate the framework on five benchmarks (MATH, MMLU, TriviaQA, SimpleQA, LiveCodeBench) across eight models from five providers. Within the deterministic threshold-cascade class, full fixed chains underperform the pairwise envelope, and optimized subsequence cascades do not deliver practically meaningful held-out gains over it. A lightweight pre-generation router exceeds the best cascade policy on four of five datasets, mainly because it avoids the cheap model's generation cost on queries sent directly to a larger model rather than because of a stronger routing signal. These results suggest that cascade performance is limited primarily by structural cost, since cascades pay the cheap model before any escalation decision, rather than by a shortage of intermediate stages.

大模型级联成本优化决策理论模型调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。