arXiv:2607.02518cs.DCcs.AI2026-07

优化长上下文代理任务的单节点推理,提升吞吐与响应速度。

GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads

论文配图:GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads
图 1 · 摘自论文原文
  • 采用分块预填充与并行策略调优模型部署
  • 请求吞吐提升10.4%,延迟降低20%以上
  • 适合长上下文、工具调用密集型应用

OpenClaw请求以长而工具增强的前缀为主,每请求输入约28k-30k token,输出500 token。服务性能由吞吐率、首次响应时间(TTFT)和尾部延迟决定。本报告研究在多模型即服务(MaaS)架构下,单一节点优化模块中GLM-5的参数调优,涵盖分块预填充、张量并行(TP)、流水线并行(PP)与请求并发。实验在双节点十六卡集群上进行。最优配置为分块预填充大小3072,tp=4,pp=4,最大运行请求数24。相比基线(2048/4/4/16),请求吞吐从0.43提升至0.48 req/s,总令牌吞吐从9029.64增至9993.23 tok/s,平均TTFT从8.98秒降至6.69秒,延迟P90从40.23秒降至32.64秒。相同硬件下,请求成本降低10.4%,每令牌成本降9.6%。结果表明最优配置具有任务特异性,大分块与深队列不单调提升性能,建议默认采用3072/tp4/pp4/max24配置。

原文摘要 · Abstract (English)

OpenClaw requests are dominated by long, tool-augmented prefixes, including system prompts, conversation history, and tool outputs fed back into the context window. For this workload, with about 28k-30k input tokens and 500 output tokens per request, serving quality is governed by throughput, TTFT, and tail latency rather than short-prompt throughput alone. This report studies GLM-5 serving-parameter tuning within a MaaS multi-model inference optimization architecture. The scope is the Single-Node Optimization block of the inference-optimization layer, where chunked prefill, tensor parallelism (TP), pipeline parallelism (PP), and request concurrency are tuned for one GLM-5 serving deployment; in this report, "Single-Node Optimization" denotes the architecture block, while experiments run on a two-node, sixteen-GPU cluster. Within the tested space, the best configuration is chunked-prefill-size=3072, tp=4, pp-size=4, and max-running-requests=24. Compared with the conservative 2048/4/4/16 baseline, it increases request throughput from 0.43 to 0.48 req/s and total token throughput from 9029.64 to 9993.23 tok/s, while reducing average TTFT from 8.98 to 6.69 s and latency P90 from 40.23 to 32.64 s. Under the same hardware footprint, this corresponds to an estimated 10.4% lower serving cost per request and 9.6% lower cost per token. The results show that the optimum is workload-specific: larger chunk sizes and deeper queueing do not monotonically improve performance. We therefore recommend 3072 / tp4 / pp4 / max24 as the default OpenClaw deployment profile.

长上下文推理优化MaaSGLM-5

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。