对比20个大模型生成代码的能耗,发现多数不如人工写的节能。
Evaluating the Energy-Efficiency of the Code Generated by LLMs
- 选取LeetCode题目,对比大模型与人工代码的能耗表现。
- 平均来看,人工代码比最优模型节能1.21倍,部分场景差超2倍。
- 动态规划等算法中,模型代码能耗最高达人工版450倍。
随着大语言模型(LLMs)生成代码质量的提升,其在软件行业的自动化代码生成应用持续增长。然而,研究普遍关注功能正确性,忽视了代码的能源效率和环境影响。本文评估了20个主流大模型在878道来自LeetCode、难度和算法类型各异的编程问题上生成代码的能耗,并与标准人工编写解法进行对比。尽管大多数情况下模型生成的代码功能正确,但其性能和能效普遍远低于人工解法。其中,DeepSeek-v3和GPT-4o生成的代码最节能,而Grok-2和Gemini-1.5-Pro最耗能。平均而言,人工解法比DeepSeek-v3高效1.17倍,比GPT-4o高效1.21倍,比Grok-2和Gemini-1.5-Pro高效超2倍。在动态规划、回溯和位运算等特定算法类别中,模型生成代码的能耗最高可达人工解法的450倍。
原文摘要 · Abstract (English)
As the quality of code generated by Large Language Models (LLMs) improves, their adoption in the software industry for automated code generation continues to grow. Researchers primarily focus on enhancing the functional correctness of the generated code while commonly overlooking its energy efficiency and environmental impact. This paper investigates the energy efficiency of the code generated by 20 popular LLMs for 878 programming problems of varying difficulty levels and diverse algorithmic categories selected from the LeetCode platform by comparing them against canonical human-written solutions. Although LLMs can produce functionally correct results in most cases, our findings show that the performance and energy efficiency of LLM-produced solutions are often far below those of human-written solutions. Among the studied LLMs, DeepSeek-v3 and GPT-4o generate the most energy-efficient code, whereas Grok-2 and Gemini-1.5-Pro are among the least energy-efficient models. On average, human-generated canonical solutions are approximately 1.17 times more energy efficient than DeepSeek-v3, 1.21 times more energy efficient than GPT-4o, and over 2 times more energy efficient than Grok-2 and Gemini-1.5-Pro. For specific algorithmic groups such as dynamic programming, backtracking, and bit manipulation, LLM-generated code can consume up to 450 times more energy than human-generated canonical solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。