arXiv:2605.11603cs.AI2026-05被引 2

让大模型推理更环保,按碳排放实时选最优模型。

GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization

论文配图:GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization
图 1 · 摘自论文原文
  • 基于碳排放约束的多目标优化,动态选择低碳模型。
  • 在保持95%延迟和准确率的前提下,碳排放显著降低。
  • 适合关注绿色AI、可持续部署的研究者与工程师。

大型语言模型(LLM)的广泛应用使请求级路由成为平衡响应质量与计算成本的关键。现有路由方法通常忽略能源使用和二氧化碳排放,而电网碳强度随时间和区域变化,不同模型能耗差异显著。为此,我们提出绿色感知路由(GAR),一种受限的多目标优化框架,在满足显式准确率下限和p95延迟服务等级目标(SLO)的前提下,最小化单请求碳排放。GAR通过数据集级下限调优实现自适应约束优化,并引入轻量级估计器预测正确性、尾部延迟与碳排放,支持无需额外推理的实时路由决策。我们提出GAR-PD,一种面向滚动碳预算的在线原始对偶算法,以及若干启发式变体,在保证高可行性覆盖的同时控制准确率下降。在涵盖7B-70B参数规模异构模型池的标准NLP基准上进行的全面实验表明,GAR在维持竞争力的准确率与p95延迟保障的同时,实现了显著的碳减排,为可持续大模型推理提供了实用且理论严谨的解决方案。

原文摘要 · Abstract (English)

The growing deployment of large language models (LLMs) makes per-request routing essential for balancing response quality and computational cost across heterogeneous model pools. Current routing methods rarely consider sustainable energy use and CO2 emissions as optimization objectives, despite grid carbon intensity varying by time and region, and models differing significantly in energy consumption. To address this gap, we introduce Green-Aware Routing (GAR), a constrained multi-objective optimization framework that minimizes per-request CO2 emissions subject to explicit accuracy floors and p95-latency service-level objectives (SLOs). GAR employs adaptive constraint optimization through per-dataset floor tuning and incorporates lightweight estimators for correctness, tail latency, and carbon emissions, enabling real-time routing decisions without additional inference passes. We present GAR-PD, a practical online primal-dual routing algorithm for rolling carbon budgets, alongside heuristic variants that achieve high feasibility coverage while limiting accuracy degradation. Comprehensive experiments across standard NLP benchmarks with heterogeneous LLM pools (7B-70B) demonstrate that GAR achieves substantial carbon reductions while maintaining competitive accuracy and p95 latency guarantees, providing a practical, theoretically grounded approach to sustainable LLM inference.

绿色AI大模型路由优化碳排放

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。