让大模型生成代码时自动提前退出，省电23%~50%。
GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation
- 用强化学习动态决定生成过程何时提前终止
- 在两个开源模型上平均节能23%-50%，准确率几乎不变
- 适合关注大模型能效、代码生成落地的开发者
大型语言模型（LLMs）正广泛应用于软件开发任务，如代码补全、修改、修复和翻译。尽管训练耗能受关注，但推理阶段因高频持续调用，能耗更甚。本文提出GREEN-CODE框架，通过强化学习训练代理，在代码生成中实现动态早停，平衡准确率、延迟与能耗。在Llama 3.2 3B和OPT 2.7B两个模型上，基于JavaCorpus和PY150数据集测试，结果表明该方法可平均降低23%-50%的能耗，同时保持较高准确性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are becoming integral to daily life, showcasing their vast potential across various Natural Language Processing (NLP) tasks. Beyond NLP, LLMs are increasingly used in software development tasks, such as code completion, modification, bug fixing, and code translation. Software engineers widely use tools like GitHub Copilot and Amazon Q, streamlining workflows and automating tasks with high accuracy. While the resource and energy intensity of LLM training is often highlighted, inference can be even more resource-intensive over time, as it's a continuous process with a high number of invocations. Therefore, developing resource-efficient alternatives for LLM inference is crucial for sustainability. This work proposes GREEN-CODE, a framework for energy-aware code generation in LLMs. GREEN-CODE performs dynamic early exit during LLM inference. We train a Reinforcement Learning (RL) agent that learns to balance the trade-offs between accuracy, latency, and energy consumption. Our approach is evaluated on two open-source LLMs, Llama 3.2 3B and OPT 2.7B, using the JavaCorpus and PY150 datasets. Results show that our method reduces the energy consumption between 23-50 % on average for code generation tasks without significantly affecting accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。