arXiv:2607.13048cs.LGcs.AI2026-07中稿 · KDD

用风险函数控制大模型调用时机,降低延迟与成本。

Uncertainty-Aware Sequential Decision Rules for Event-Triggered LLM Invocation in Streaming Systems

论文配图:Uncertainty-Aware Sequential Decision Rules for Event-Triggered LLM Invocation in Streaming Systems
图 1 · 摘自论文原文
  • 基于风险的序贯决策框架,动态决定何时调用大模型。
  • 实测在1600次诊断中92.9%达到可信度≥0.75,误报率低。
  • 适合需要高可靠性与低延迟的实时流系统应用。

流式推理系统日益将轻量快速模型与成本高昂的大语言模型(LLM)结合,以实现丰富的语义理解。何时调用LLM这一核心问题缺乏正式处理。本文将其建模为基于风险的序贯停止问题:当观测历史上的风险函数超过阈值时触发调用。在此框架下,证明了六项结论:排除触发振荡的最小事件间隔;通过平滑贴合证明阈值策略最优;在参数估计下近似SPRT保证;平稳流下遗憾为O(sqrt(T log T)),存在C_T个变化点时扩展至O(sqrt((C_T + 1) T log T));在线梯度下降求解自适应阈值收敛速度O(1/sqrt(T));以及校准到漏报率的转移不等式。经典触发机制如事件触发、最优停止、SPRT、CUSUM和贝叶斯触发均可视为该框架特例。在真实大模型调用的涡轮发动机退化数据(CMAPSS)上,验证了理论假设,剖析了风险函数设计,对比六种基线(含RouteLLM风格路由器与上下文博弈),分析成本敏感性与大模型失效模式。结果表明遗憾为次线性,所有合理触发器α < 1;诊断质量高,1600次诊断中92.9%的接地分数≥0.75;且以异常得分驱动的风险函数在帕累托AUC上优于其他方案约一个数量级。

原文摘要 · Abstract (English)

Streaming inference pipelines increasingly pair lightweight fast models with Large Language Models (LLMs) that provide rich semantic understanding at substantial cost. The central question of when to invoke the LLM has received limited formal treatment. We cast this as a risk-based sequential stopping problem, where a trigger policy fires when a risk functional over the observation history exceeds a threshold. Within this framework, we prove six results: a minimum inter-event time bound excluding trigger chattering; optimality of threshold policies via smooth pasting; approximate SPRT guarantees under estimated parameters; O(sqrt(T log T)) regret for stationary streams, extending to O(sqrt((C_T + 1) T log T)) under C_T changepoints; O(1/sqrt(T)) convergence of online gradient descent for adaptive thresholds; and a calibration-to-miss-rate transfer inequality. Several classical trigger families, including event-triggered, optimal stopping, SPRT, CUSUM, and Bayesian triggers, can be expressed as special cases of this framework. On turbofan degradation data (CMAPSS) with real LLM calls, we empirically verify the theoretical assumptions, ablate the risk function design, compare against six baselines including a RouteLLM-style router and contextual bandits, and analyze cost sensitivity and LLM failure modes. The results confirm sublinear regret, with alpha < 1 for all principled triggers; high diagnostic quality, with 92.9 percent of 1600 LLM diagnoses reaching grounding score >= 0.75 under our rubric; and that anomaly-score-driven risk functions dominate alternatives by roughly an order of magnitude on the Pareto AUC.

大模型调度流式推理决策优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。