用图神经网络精准预测大模型推理的碳排放，兼顾不同硬件和请求配置。
LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferences
- 基于图神经网络建模推理过程中的预填充与解码阶段特征。
- 相比传统方法，预测误差显著降低，支持多种硬件与请求组合。
- 适合关注绿色AI、云服务碳足迹评估的研究者与开发者。
大型语言模型（LLM）在其生命周期中，推理阶段产生的碳足迹远超训练阶段。推理请求在批处理大小、提示长度和生成标记数量上差异显著，而云服务商则采用不同类型的GPU及其数量来满足不同的精度与延迟服务目标。因此，在执行前快速准确地估算基于推理请求与硬件配置组合的碳影响，对用户和云服务商都至关重要。由于推理阶段的计算量（FLOPS）利用率较低且高度可变，导致此前基于公式的模型不够准确。此外，现有的机器学习预测方法要么精度不足，要么需要大量训练数据，因其未能充分建模预填充与解码阶段的区别，忽略硬件特异性，且对罕见推理配置采样效率低。本文提出 exttt{LMMCO2}，一种基于图神经网络（GNN）的模型，显著提升了对LLM推理碳足迹的预测准确性，优于以往方法。
原文摘要 · Abstract (English)
Throughout its lifecycle, a large language model (LLM) generates a substantially larger carbon footprint during inference than training. LLM inference requests vary in batch size, prompt length, and token generation number, while cloud providers employ different GPU types and quantities to meet diverse service-level objectives for accuracy and latency. It is crucial for both users and cloud providers to have a tool that quickly and accurately estimates the carbon impact of LLM inferences based on a combination of inference request and hardware configurations before execution. Estimating the carbon footprint of LLM inferences is more complex than training due to lower and highly variable model FLOPS utilization, rendering previous equation-based models inaccurate. Additionally, existing machine learning (ML) prediction methods either lack accuracy or demand extensive training data, as they inadequately handle the distinct prefill and decode phases, overlook hardware-specific features, and inefficiently sample uncommon inference configurations. We introduce \coo, a graph neural network (GNN)-based model that greatly improves the accuracy of LLM inference carbon footprint predictions compared to previous methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。