用推理时间估算API模型的能耗，帮助用户了解大模型的碳足迹。
This Is Taking Too Long -- Investigating Time as a Proxy for Energy Consumption of LLMs
- 以推理时长为代理指标，估算API调用的大模型能耗
- 实测验证：时长与能耗高度相关，误差可控
- 适合关心模型碳排放的开发者和研究者
大型语言模型（LLMs）的能源消耗日益引发对环境稳定与资源使用的担忧。然而，当模型通过API调用时，其能耗对用户而言仍不透明——这是一个信息由服务商单方面披露的黑箱。本文研究了推理时长作为估算API类大模型能耗的代理指标。通过将云端模型的时长测量结果与本地部署版本的实际能耗对比，验证了时长与能耗之间的强相关性。实验表明，利用推理时间可有效推断出基于GPU的API模型能耗，为用户提供了一种可行的能耗估算手段。本研究旨在提升用户对大模型碳足迹的认知，尤其适用于关注可持续性的终端使用者。
原文摘要 · Abstract (English)
The energy consumption of Large Language Models (LLMs) is raising growing concerns due to their adverse effects on environmental stability and resource use. Yet, these energy costs remain largely opaque to users, especially when models are accessed through an API -- a black box in which all information depends on what providers choose to disclose. In this work, we investigate inference time measurements as a proxy to approximate the associated energy costs of API-based LLMs. We ground our approach by comparing our estimations with actual energy measurements from locally hosted equivalents. Our results show that time measurements allow us to infer GPU models for API-based LLMs, grounding our energy cost estimations. Our work aims to create means for understanding the associated energy costs of API-based LLMs, especially for end users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。