提出BPW框架，精准保留大模型服务中各类瓶颈的证据，提升测试效率与诊断可靠性。
Diagnose Before You Compress: Prediction-Independent Bottleneck Witness Refinement for LLM Serving Traces
- 通过无响应特征和系统测量初选可能暴露瓶颈的请求
- 按覆盖优先级构建多组件可复用请求序列，确保各瓶颈被充分验证
- 基于直接系统测量生成独立于预测的瓶颈标签，避免评估循环
生产环境的大模型服务生成数百万种多样请求，跨配置完整追踪成本高昂。现有压缩方法主要保留工作负载分布或代表性请求，但揭示瓶颈的工作负载往往稀少且不具代表性。此外，一个组件的证据无法弥补另一组件缺失的证据，而以预测瓶颈作为真实标签会导致评估循环。因此需在不依赖代表性的情况下，保留每个瓶颈组件的证据。本文提出瓶颈保持型见证（BPW），一种质量约束下的紧凑、可诊断的推理重现框架。首先使用无响应特征和闭源侧测量进行工作负载候选提名，识别可能暴露调度、预填充、解码或KV缓存瓶颈的请求；接着通过覆盖率优先的序列构建，将多组件提案组织为可复用的超边并优先处理未覆盖维度；最后通过瓶颈真值验证，仅基于目标系统直接测量生成独立于预测的标签，确定满足每组件双证据要求的最早前缀。在BurstGPT、ServeGen和Mooncake上的实验表明，BPW以紧凑请求集达到验证门限，优于16种策略，均前宏F1和WBRC-AUC分别提升2.3%和16.3%。阶段解析与敏感性分析验证了三阶段的独立贡献与局部稳定性。代码已开源。
原文摘要 · Abstract (English)
Production LLM serving generates millions of diverse requests, making full-trace replay across serving configurations increasingly expensive. Existing trace reduction methods mainly preserve workload distributions or representative requests, but bottleneck-revealing workloads may be rare and non-representative. Moreover, evidence for one component cannot compensate for missing evidence in another, while using predicted bottlenecks as target truth creates circular evaluation. These limitations make it necessary to preserve evidence for every bottleneck component rather than rely on workload representativeness alone. We propose Bottleneck-Preserving Witnessing (BPW), a quality-constrained framework for compact and diagnostically reliable LLM serving replay suites. BPW first performs Workload Candidate Nomination using response-blind workload features and closed source-side measurements. This stage identifies workloads that may expose scheduler, prefill, decode, or KV-cache bottlenecks. Coverage-Priority Sequence Construction then organizes multi-component proposals as reusable hyperedges and prioritizes weak and uncovered dimensions. Finally, Bottleneck Truth Verification derives prediction-independent labels solely from direct target-system measurements. The verified results determine the earliest prefix satisfying the direct two-witness requirement for every component. Experiments on BurstGPT, ServeGen, and Mooncake show that BPW reaches the verified gate with a compact workload set and outperforms 16 policies, achieving relative improvements of 2.3% and 16.3% in Mean prefix Macro-F1 and WBRC-AUC, respectively. Stage-resolved and sensitivity analyses confirm the distinct contributions and local stability of its three stages. Our code is publicly available at https://github.com/llmllmllm/BPW
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。