arXiv:2602.05712cs.SEcs.AI2026-02被引 4

分析大模型推理能耗,发现预填充阶段影响生成阶段耗能,抑制冗余输出可省电超89%。

Towards Green AI: Decoding the Energy of LLM Inference in Software Development

  • 分预填充与解码阶段分析能耗,揭示两阶段关联性
  • 预填充成本上升使解码能耗增加1.3%至51.8%
  • 抑制冗余输出可节能44%~89%,且不降生成质量

背景:AI辅助开发工具日益依赖大语言模型(LLM),带来显著计算与能耗负担。理解并降低LLM推理能耗对可持续软件开发至关重要。目标:本研究开展基于阶段的推理能耗分析,区分(1)预填充阶段——模型处理输入并构建内部表示;(2)解码阶段——利用存储状态逐个生成输出词元。方法:评估六款6B-7B和四款3B-4B的Transformer模型,在HumanEval(代码生成)与LongBench(代码理解)两个代码相关基准上进行测试。结果:两类模型在各阶段均呈现明显能耗差异。预填充成本升高会放大解码阶段每词元能耗,增幅为1.3%至51.8%。此外,三款模型存在冗余生成行为(babbling),导致输出内容过多,浪费能源。通过实施冗余抑制策略于代码生成任务,实现44%至89%的能耗降低,且不影响生成准确性。结论:预填充阶段影响解码能耗,后者主导总能耗;抑制冗余生成可实现高达89%的节能。因此,降低推理能耗需同时控制冗余行为与预填充对解码的影响。

原文摘要 · Abstract (English)

Context: AI-assisted tools are increasingly integrated into software development workflows, but their reliance on large language models (LLMs) introduces substantial computational and energy costs. Understanding and reducing the energy footprint of LLM inference is therefore essential for sustainable software development. Objective: In this study, we conduct a phase-level analysis of LLM inference energy consumption, distinguishing between the (1) prefill, where the model processes the input and builds internal representations, and (2) decoding, where output tokens are generated using the stored state. Method: We investigate six 6B-7B and four 3B-4B transformer-based models, evaluating them on code-centric benchmarks HumanEval for code generation and LongBench for code understanding. Results: Our findings show that, within both parameter groups, models exhibit distinct energy patterns across phases. Furthermore, we observed that increases in prefill cost amplify the energy cost per token during decoding, with amplifications ranging from 1.3% to 51.8% depending on the model. Lastly, three out of ten models demonstrate babbling behavior, adding excessive content to the output that unnecessarily inflates energy consumption. We implemented babbling suppression for code generation, achieving energy savings ranging from 44% to 89% without affecting generation accuracy. Conclusion: These findings show that prefill costs influence decoding, which dominates energy consumption, and that babbling suppression can yield up to 89% energy savings. Reducing inference energy therefore requires both mitigating babbling behavior and limiting impact of prefill on decoding.

绿色AILLM能耗代码生成节能优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。