提出可精确计算请求级能耗的框架,解决批量推理中能耗分配不公平问题。
Request-Level Energy Attribution for Batched LLM Serving

- 基于重现性重放与GPU功耗数据,计算每个请求的精确谢尔比能耗份额
- 在静态和连续批处理下,误差降低至0.116和0.177,优于传统比例分配
- 适用于数据中心碳核算、成本分摊,适合需公平能耗分配的系统
批量大模型服务虽提升吞吐量,却使能耗归因复杂化。现有方法仅提供模型、阶段或令牌级能耗,缺乏请求级真实基准。本文提出JouleShare框架:离线通过vLLM重现请求子集,结合GPU功耗数据,计算每个请求的精确谢尔比(Shapley)能耗份额;再训练轻量级校准模型JCalib,用低成本请求特征在线预测该份额。在16组模型/工作负载实验中,基于令牌比例的分配在静态批处理下平均偏离真实值0.440(归一化L1),连续批处理下为0.458,三款数据中心GPU均复现此差距。JCalib将误差降至0.116(静态)和0.177(连续),优于无法在线获取的基线测量。采样谢尔比扩展了基准范围,且单次离线校准始终为最准确部署规则。结果表明,令牌比例不能可靠反映批量执行下的边际能耗,而经测量的谢尔比基准可有效校准低成本特征以实现更公平分配。
原文摘要 · Abstract (English)
Batched LLM serving improves throughput but complicates energy accounting. GPU power telemetry is aggregate, whereas sustainability reporting, chargeback, and workload analysis often require request-level energy charges. Existing inference-energy benchmarks report model-, phase-, or token-level energy, and recent carbon-accounting work motivates Shapley fairness conceptually. Neither provides measured request-level ground truth, so how far the accounting rules used in practice deviate from a fair allocation has remained unknown. We present JouleShare, an attribution framework with two components. An offline harness establishes this ground truth by replaying request subsets under vLLM with a reproducible protocol, integrating GPU power telemetry, and computing exact Shapley energy for each request. A lightweight calibration model, JCalib, then learns to predict Shapley shares from cheap request features for use at serving time. Across 16 model/workload runs, token-proportional attribution differs from exact Shapley by 0.440 normalized L1 on average under static batching and by 0.458 under continuous batching, a gap that reproduces across three data-center GPUs. JCalib reduces this error to 0.116 under static batching and 0.177 under continuous batching, below even a standalone-measurement baseline that is unavailable online, while preserving exact batch-energy efficiency. Sampled Shapley extends the measured reference to larger group sizes, where the gap persists and a single offline calibration remains the most accurate deployable rule. The results show that token attribution is not a reliable proxy for marginal energy under batched execution, and that measured Shapley ground truth can calibrate low-cost request features toward fairer attribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。