揭示了持续运行模型的隐性能耗，发现待机耗电主要来自上下文开销而非显存占用。
The Model Parking Tax: Quantifying the Hidden Energy Cost of Always-On GPU Model Deployment
- 测量三种GPU在不同显存技术下的待机功耗，发现空闲状态分段恒定。
- 上下文开销贡献超98%的待机能耗，显存每增加1GB仅多耗0.02瓦以下。
- 提出冷启动能耗平衡模型，适用于高并发、低延迟场景的部署决策。
AI推理行业长期将模型常驻于GPU内存以避免冷启动延迟，隐性将空闲功耗视为就绪成本。然而该成本结构从未在跨架构层面被实证分解。本文首次对三类跨越三种显存技术的GPU（NVIDIA H100/HBM3/80GB、A100/HBM2e/80GB、L40S/GDDR6/48GB）进行跨架构待机功耗测量，结合18天生产环境数据（335,267样本，14张H100 GPU）与受控剂量反应实验。结果表明，所有设备上待机功耗均为分段常数：CUDA上下文触发离散DVFS跃迁，额外消耗+26-66 W（HBM架构空闲26-50 W，GDDR6为66 W），而显存边际影响低于测量敏感度（|β| < 0.02 W/GB）。CUDA上下文贡献>98%的待机成本，无论显存技术。通过真实HuggingFace模型Qwen2.5-7B验证，各设备空显存张量间差异<0.5 W，同时捕捉到模型加载过程中的冷启动功耗曲线。推导出冷启动盈亏平衡模型，显示能效最优行为取决于请求到达率与加载延迟，而非模型大小，盈亏区间为1-5分钟。结果揭示所有测试架构的共性约束：待机带上下文功耗由DVFS状态决定，而非显存占用。
原文摘要 · Abstract (English)
The AI inference industry keeps models loaded in GPU memory around the clock to avoid cold-start latency, implicitly treating idle power as a fixed cost of readiness. Yet the structure of this cost has never been empirically decomposed - and never across GPU architectures. We present the first cross-architecture measurement of idle GPU power as a function of VRAM allocation, combining 18 days of production telemetry (335,267 samples, 14 H100 GPUs) with controlled dose-response experiments on three GPU architectures spanning three memory technologies: NVIDIA H100 (HBM3, 80 GB), A100 (HBM2e, 80 GB), and L40S (GDDR6, 48 GB). We observe that idle power is piecewise constant on all three architectures: the CUDA context forces a discrete DVFS transition consuming +26-66 W over bare idle (26-50 W on HBM architectures, 66 W on GDDR6), while the marginal VRAM effect is bounded below measurement relevance ($|β| < 0.02$ W/GB) on every device tested. The CUDA context accounts for >98% of the parking tax regardless of memory technology. We validate this finding with a real HuggingFace model (Qwen2.5-7B) on all three architectures, confirming <0.5 W difference from empty tensors on every device, and capture cold-start power profiles during model loading. We derive a cold-start breakeven model showing energy-optimal behavior depends on request arrival rate and loading latency - not model size - with breakeven intervals of 1-5 minutes. Our results identify a constraint consistent across all tested architectures: idle-with-context power is determined by DVFS state, not memory occupancy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。