让大模型服务调优更稳定，避免崩溃且结果更一致。
SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving
- 用崩溃信息指导搜索，把失败当作有用数据
- 相同预算下更快找到高效配置，结果波动小4.4倍
- 适合追求稳定部署的工程团队使用
在延迟服务等级目标（SLO）约束下部署大语言模型是一项高度依赖配置的系统问题，其搜索空间异常易出错：许多合理配置会直接崩溃或无法满足用户可见的延迟目标，而传统黑盒优化器将这些失败视为无效试验。我们提出SLO-Guard，一种针对vLLM服务的抗崩溃自动调优方法，将崩溃视为第一类观测结果。SLO-Guard结合了以可行解优先的热预算退火（TBA）探索阶段与冷启动的树结构帕累托估计器（TPE）利用阶段；交接时复现全部探索历史，包括将崩溃编码为极端约束违反。我们还引入配置修复模块、基于GPU的KV缓存内存防护机制及四类崩溃分类体系。在NVIDIA A100 40GB上使用vLLM 0.19运行Qwen2-1.5B进行评估。在预设五种子实验中，SLO-Guard与均匀随机搜索均实现75/75可行性，零崩溃，最佳延迟无显著差异（双侧Mann-Whitney检验p=0.84）。SLO-Guard的优势在于预算一致性：在快速服务区间内执行更多有效试验（10.20 vs. 7.40/15；单侧p=0.014），交接后一致性更高（0.876 vs. 0.539；p=0.010）。并发负载下，SLO-Guard跨种子的最佳延迟标准差仅为随机搜索的4.4倍（2.26毫秒 vs. 10.00毫秒）。独立顺序调度测试验证了结果稳健性。核心观点并非找到更优最终配置，而是固定预算下一旦进入快速模式，能更可预测地完成调优。
原文摘要 · Abstract (English)
Serving large language models under latency service-level objectives (SLOs) is a configuration-heavy systems problem with an unusually failure-prone search space: many plausible configurations crash outright or miss user-visible latency targets, and standard black-box optimizers treat these failures as wasted trials. We present SLO-Guard, a crash-aware autotuner for vLLM serving that treats crashes as first-class observations. SLO-Guard combines a feasible-first Thermal Budget Annealing (TBA) exploration phase with a warm-started Tree-structured Parzen Estimator (TPE) exploitation phase; the handoff replays all exploration history, including crashes encoded as extreme constraint violations. We additionally contribute a configuration-repair pass, a GPU-aware KV-cache memory guard, and a four-category crash taxonomy. We evaluate SLO-Guard on Qwen2-1.5B served with vLLM 0.19 on an NVIDIA A100 40GB. Across a pre-specified five-seed study, both SLO-Guard and uniform random search attain 75/75 feasibility with zero crashes under the corrected concurrent harness, and are statistically tied on best-achieved latency (Mann-Whitney two-sided p=0.84). SLO-Guard's advantage is in budget consistency: more trials in the fast-serving regime (10.20 vs. 7.40 out of 15; one-sided p=0.014) and higher post-handoff consistency (0.876 vs. 0.539; p=0.010). Under concurrent load, SLO-Guard's cross-seed standard deviation on best latency is 4.4x tighter than random search's (2.26 ms vs. 10.00 ms). A harness-replication analysis shows that the consistency findings survive an independent sequential-dispatch measurement condition. The central claim is not that SLO-Guard finds a better final configuration, but that it spends a fixed tuning budget more predictably once the fast regime has been found.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。