arXiv:2608.23986math.OCcs.AI2026-08

大模型推理降级看似省钱,实则因失败重试和用户流失反而更耗资源。

The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem

  • 用供应链视角建模推理分配,将失败重试与用户流失量化为成本
  • 降级虽省算力却增负载,容量受限时反使整体成本上升
  • 提出智能分级调度策略,按客户类别动态分配推理资源

大型语言模型服务商面临算力约束,应对拥堵的通用做法是降低服务:将请求转至小模型、减少推理深度、截断上下文。行业会计认为此举可节省成本,但我们证明这一计算错误——其仅在客户购买答案时定价,而劣化答案存在失败概率,失败后要么重试(加剧系统高峰期负载),要么导致用户流失(损害长期价值,此成本未被成本仪表盘显示)。我们以三个经典模型构建推理分配框架:一个库存缺货成本为流失用户终身价值的报童模型;一个重试乘数由不满情绪驱动的几何模型;以及一个由重试内生决定到达率的双状态瞬态队列模型。静态分析表明,在某些可测区间内,更廉价模型虽单位满意回答能耗更低,但单位满意回答占用容量更高,当容量成为瓶颈时,折扣效应反转。动态分析显示,高峰时触发的动态限流可能越过临界阈值,产生比其削减更多的新流量;若释放规则低于降级平衡点,则短暂峰值会演变为永久降级状态。针对异构用户,限流本质是重试放大负载下的运输问题,最优策略按关键比率逐类分配智能资源,其对偶即‘智能影子价格’,可按类别与小时精确计价,闭式解可在毫秒内计算。随机分析强化而非削弱结论:临界边界具有预测宽度,噪声会惩罚依赖该边界的反应式策略。因此,在拥塞时,限流并非成本杠杆,而是需求杠杆。

原文摘要 · Abstract (English)

Large language model providers are compute constrained, and their universal response to congestion is to degrade service: route queries to smaller models, cut reasoning effort, truncate context. The industry's accounting says this saves money. We show the accounting is wrong, because it prices a query when the customer buys an answer. A degraded answer fails with some probability, and a failed answer either returns as a retry, inflating arrivals when the system is most loaded, or departs as churn, destroying lifetime value on a ledger no cost dashboard displays. We model inference allocation with three classical primitives: a newsvendor whose stockout cost is churned lifetime value, a geometric retry multiplier in which the recycled product is dissatisfaction, and a two-regime transient queue whose arrival rate is made endogenous by retries. Statically, there is a nonempty, measurable regime in which a cheaper model saves energy per satisfied answer while consuming strictly more capacity per satisfied answer, so the discount inverts exactly when capacity binds. Dynamically, a reactive throttle fired during a surge can cross an ignition threshold beyond which it manufactures more traffic than it sheds, and a release rule set below the degraded equilibrium converts a transient surge into a permanent degraded regime. With heterogeneous customers, throttling is a transportation problem in retry-inflated load whose optimal policy rations intelligence by critical ratio, class by class, and whose dual, the shadow price of intelligence, prices a marginal query by class and by hour; closed-form trajectories make it computable in milliseconds. Stochastic analysis sharpens rather than erodes the thesis: the ignition boundary acquires a predicted width, and noise punishes the reactive policy that parks the system against it. Under congestion, throttling is not a cost lever but a demand lever.

大模型推理服务降级资源调度随机优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。