arXiv:2607.04951cs.DCcs.CL2026-07

用语言特征提前预测文本负载,避免边缘设备崩溃。

When Words Predict Workload

  • 通过16维文本结构向量+XGBoost预测请求风险等级
  • 实测误调度率降至0.095,远低于基线的0.849
  • 适合处理法律文书等结构僵化的高风险文本场景

标准分布式大模型调度器依赖静态词元数或滚动延迟均值,在受法规约束的文本上易失效。针对由《欧洲专利公约》第84条管辖的专利申请,语言僵化使人类与机器创作难以区分。飞行中解决此歧义需动态扩展多模型集成,引发不可预测的键值缓存和权重分配激增,导致消费级边缘GPU显存饱和并严重溢出崩溃。为防止硬件瘫痪,我们提出基于CPU的语义资源预测(LRF)网关。该网关提取16维文本结构向量,利用XGBoost预测陷阱带归属概率($\Pesc$),并与实时延迟遥测计算的动态路由阈值($\Tauroute(t)$)比较。请求在分配边缘显存前即被安全路由至本地Qwen2.5-7B或远程对比集成(Qwen2.5 7B + 32B)。在6000次请求的现场测试中,LRF网关将误调度率($R_{\mathrm{mis}}$)降至0.087–0.095,较词元计数基线降低一个数量级;峰值边缘显存稳定在4.82 GiB(低于8 GiB上限),跨27倍广域网延迟波动保持安全。预测器实现现场AUROC 0.84,动态阈值控制器相较静态阈值减少8.2%误调度。

原文摘要 · Abstract (English)

Standard distributed \ac{llm} schedulers rely on static token counts or rolling latency averages, making them susceptible to failures on statutorily constrained text. On \ac{epo} claims governed by Article 84 \ac{epc}, linguistic rigidity makes human and machine authorship statistically indistinguishable. Resolving this ambiguity mid-flight forces dynamic multi-model ensemble expansion, triggering unpredictable KV-cache and weight-allocation spikes that saturate consumer-grade edge GPU VRAM and cause severe \ac{oom} crashes. To prevent hardware collapse, we propose a CPU-side Linguistic Resource Forecasting (LRF) gateway. The gateway extracts a 16-dimensional text-structure vector and applies an XGBoost predictor to forecast trap-band membership. The resulting escalation probability ($\Pesc$) is evaluated against a dynamic, closed-form routing threshold ($\Tauroute(t)$) computed via real-time latency telemetry. Requests are safely routed to either a local Qwen2.5-7B edge worker or a remote contrastive ensemble (Qwen2.5 7B + 32B) on an NVIDIA H100 \emph{before} any edge GPU memory is allocated. In a 6,000-request live trial, the LRF gateway reduced the operational misroute fraction ($R_{\mathrm{mis}}$) to $0.087$--$0.095$, an order of magnitude below the token-count baseline ($0.849$). Peak edge VRAM remained safely bounded at $\SI{4.82}{\gibi\byte}$ (under the $\SI{8}{\gibi\byte}$ ceiling) across a $27\times$ variation in \ac{wan} delay. The predictor achieved a live-trial AUROC of $0.84$, and the dynamic $\Tauroute(t)$ controller yielded an $8.2\%$ relative reduction in misroutes compared to an equivalent static threshold.

调度优化边缘计算大模型推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。