小模型生成代码更省电，正确率接近大模型。
Energy-Aware Code Generation with LLMs: Benchmarking Small vs. Large Language Models for Sustainable AI Programming
- 对比150道力扣题，用小模型生成代码
- 小模型在52%题目中能耗低于大模型
- 适合关注绿色编程的开发者
大型语言模型（LLMs）广泛用于代码生成，但其高算力需求导致高能耗与碳排放。本研究评估了专为代码生成设计的开源小型语言模型（SLMs），并与大模型及人工编写代码进行对比。实验涵盖LeetCode中150道按难度均分的题目，比较了StableCode-3B、StarCoderBase-3B、Qwen2.5-Coder-3B-Instruct三个小模型，以及GPT-4.0和DeepSeek-Reasoner两个大模型。评估指标包括运行时间、内存占用、能耗和正确性。以人工代码为基准，结果显示：大模型在所有难度下正确率最高，但小模型在输出正确时通常更节能，在超过52%的问题中能耗不高于大模型。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are widely used for code generation. However, commercial models like ChatGPT require significant computing power, which leads to high energy use and carbon emissions. This has raised concerns about their environmental impact. In this study, we evaluate open-source Small Language Models (SLMs) trained explicitly for code generation and compare their performance and energy efficiency against large LLMs and efficient human-written Python code. The goal is to investigate whether SLMs can match the performance of LLMs on certain types of programming problems while producing more energy-efficient code. We evaluate 150 coding problems from LeetCode, evenly distributed across three difficulty levels: easy, medium, and hard. Our comparison includes three small open-source models, StableCode-3B, StarCoderBase-3B, and Qwen2.5-Coder-3B-Instruct, and two large commercial models, GPT-4.0 and DeepSeek-Reasoner. The generated code is evaluated using four key metrics: run-time, memory usage, energy consumption, and correctness. We use human-written solutions as a baseline to assess the quality and efficiency of the model-generated code. Results indicate that LLMs achieve the highest correctness across all difficulty levels, but SLMs are often more energy-efficient when their outputs are correct. In over 52% of the evaluated problems, SLMs consumed the same or less energy than LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。