LLMs生成的代码能效不如人类专家，尤其在服务器上差16%。
Generating Energy-Efficient Code via Large-Language Models -- Where are we now?
- 用6个主流LLM和4种提示策略生成代码，对比人写与专家代码
- 服务器上人类代码省电16%，树莓派省3%，但电脑上LLM反超25%
- 代码能效需人工专家指导，当前LLM仍难替代绿色编程经验
背景：大语言模型（LLMs）已广泛融入开发流程。目标：实证评估LLMs生成的Python代码在能耗上与人类编写代码及绿色软件专家代码的差异。方法：在EvoEval基准的9个编程问题上，使用6个主流LLM配合4种提示技术生成363个解决方案，并与人工代码对比；在服务器、个人电脑和树莓派三类硬件上测试，总耗时约881小时（36.7天）。结果：人类代码在服务器上比LLM平均节能16%，在树莓派上节能3%；但在个人电脑上，部分LLM代码反而比人类代码节能25%。提示策略对节能效果无稳定提升作用，最有效提示因硬件平台而异。绿色软件专家编写的代码在所有平台、所有情况下均优于各LLM，节能幅度达17%至30%。结论：尽管LLMs具备较强的代码生成能力，但尚未有生成代码超越经验丰富的绿色软件开发者，表明当前仍需依赖人工专业知识来实现高效的能源节约型编程。
原文摘要 · Abstract (English)
Context. The rise of Large Language Models (LLMs) has led to their widespread adoption in development pipelines. Goal. We empirically assess the energy efficiency of Python code generated by LLMs against human-written code and code developed by a Green software expert. Method. We test 363 solutions to 9 coding problems from the EvoEval benchmark using 6 widespread LLMs with 4 prompting techniques, and comparing them to human-developed solutions. Energy consumption is measured on three different hardware platforms: a server, a PC, and a Raspberry Pi for a total of ~881h (36.7 days). Results. Human solutions are 16% more energy-efficient on the server and 3% on the Raspberry Pi, while LLMs outperform human developers by 25% on the PC. Prompting does not consistently lead to energy savings, where the most energy-efficient prompts vary by hardware platform. The code developed by a Green software expert is consistently more energy-efficient by at least 17% to 30% against all LLMs on all hardware platforms. Conclusions. Even though LLMs exhibit relatively good code generation capabilities, no LLM-generated code was more energy-efficient than that of an experienced Green software developer, suggesting that as of today there is still a great need of human expertise for developing energy-efficient Python code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。