用资源底限分析代替盲目调参,提升大模型服务优化效率
Think Before You Grid-Search: Floor-First Triage for LLM Serving
- 通过五维资源向量建模解码过程,计算乐观与悲观底限区间
- 实测值落在区间内可提前判断资源瓶颈,仅在残差超阈值时启用剖析器
- 支持零依赖部署,适合生产环境的智能调度与架构决策
大模型服务优化常依赖大量配置测试和重型剖析工具。本文提出反向策略:估算应是剖析的前置分析层,否则优化退化为网格搜索。提出Floor First方法,将每个解码步骤建模为五维资源向量(HBM字节、浮点操作、网络字节、网络消息、KV容量),对资源内求和得悲观底限,跨资源最大化得乐观底限。实测值位于[最大值, 总和]区间内即可预判资源重叠质量,无需打开剖析器;仅当残差超过阈值才升级剖析。部署方案通过‘资源墙’先触达顺序比较,而非点测基准。该方法具有组合性:新注意力或状态空间模块只需声明一个组件即可接入。案例分析基于16块NVIDIA H20 GPU上的DeepSeek-V3.2风格671B MoE/MLA模型,其约74 FLOP/byte的峰值性能(相较H100的590)表明其高度解码优化。底限分析显示TP16解码受KV容量限制,最多支持约70个并发8K请求;稀疏注意力消除KV带宽项但无法突破容量墙;而EP16+DP注意力布局虽加重同批权重流量,但将容量墙提升至约644,实现数量级增长——尽管单流延迟上TP仍快2.4倍。因此布局选择是运行点的可计算函数,解释了相同硬件下生产部署采用相反注意力布局的原因。
原文摘要 · Abstract (English)
LLM serving optimization typically benchmarks many configurations and reaches for heavy profilers when latency targets are missed. We argue for the reverse discipline: estimation is the analytical layer of profiling -- without it, optimization degenerates to grid search. Floor First is a residual-driven triage workflow. Each decode step is modeled as a five-dimensional resource vector (HBM bytes, FLOPs, network bytes, network messages, KV capacity); summing within a resource and maximizing across resources gives an optimistic floor, the plain sum a pessimistic one. Where a measurement lands inside this [max, sum] interval reads out overlap quality before any profiler is opened, and profilers escalate only on residuals above a stated threshold. Deployment alternatives are compared by wall ordering -- which resource wall binds first as load grows -- rather than by point benchmarks. The account is compositional: new attention or state-space variants enter by declaring one module, and the workflow ships as a zero-dependency calculator plus an agent skill that enforces the discipline in agentic optimization loops. As a case study we analyze a DeepSeek-V3.2-style 671B MoE/MLA model on 16 NVIDIA H20 GPUs, whose ridge point of ~74 FLOP/byte (vs ~590 for H100) makes it an extreme decode-oriented part. The floors show TP16 decoding is KV-capacity-limited to ~70 concurrent 8K requests; sparse attention removes the KV-bandwidth term but not the capacity wall; an EP16+DP-attention layout accepts slightly worse same-batch weight traffic for an order-of-magnitude higher capacity wall (~644) -- while single-stream latency favors TP by 2.4x. The layout judgment is thus a computable function of the operating point, explaining why production deployments on identical hardware have shipped opposite attention layouts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。