对比大模型与人类优化代码，发现大模型虽思路多样但节能效果不佳。
Can We Make Code Green? Understanding Trade-Offs in LLMs vs. Human Code Optimizations
- 对比GPT、Llama等大模型与资深开发者在400个Matlab脚本上的优化方案
- 大模型建议的2176项优化未降低能耗或执行时间，反而恶化内存使用
- 揭示向量化、预分配等优化主题存在能效-内存权衡，适合关注绿色编程的开发者
快速的技术演进加速了各领域软件开发，推动全球碳排放上升。尽管近期大型语言模型（LLMs）宣称可帮助开发者优化代码以提升性能与能效，其在真实场景中的有效性仍待验证。本文研究了LLMs在减少实际项目环境足迹方面的表现，聚焦广泛用于科研与工程领域的Matlab代码。我们分析了100个顶级GitHub仓库中400个脚本的能源优化，评估了包括GPT-3、GPT-4、Llama、Mixtral在内的主流大模型及一位资深Matlab开发者提出的2,176项优化建议，涵盖能耗、内存使用、执行时间与代码正确性。将优化映射至13个高层次主题后发现，大模型提出多样化改进，包括可读性、可维护性、内存管理与错误处理；而人类开发者则忽略了部分并行处理与错误处理问题。然而统计测试显示，这些能源优化意外加剧了内存消耗，对执行时间与能耗均无显著改善。定性分析揭示向量化、预分配等主题是能效-时间权衡的关键来源。随着大模型日益普及于现代软件开发，本研究呼吁重视对常见编码实践的绿色评估。
原文摘要 · Abstract (English)
The rapid technological evolution has accelerated software development for various domains and use cases, contributing to a growing share of global carbon emissions. While recent large language models (LLMs) claim to assist developers in optimizing code for performance and energy efficiency, their efficacy in real-world scenarios remains under exploration. In this work, we explore the effectiveness of LLMs in reducing the environmental footprint of real-world projects, focusing on software written in Matlab-widely used in both academia and industry for scientific and engineering applications. We analyze energy-focused optimization on 400 scripts across 100 top GitHub repositories. We examine potential 2,176 optimizations recommended by leading LLMs, such as GPT-3, GPT-4, Llama, and Mixtral, and a senior Matlab developer, on energy consumption, memory usage, execution time consumption, and code correctness. The developer serves as a real-world baseline for comparing typical human and LLM-generated optimizations. Mapping these optimizations to 13 high-level themes, we found that LLMs propose a broad spectrum of improvements--beyond energy efficiency--including improving code readability and maintainability, memory management, error handling while the developer overlooked some parallel processing, error handling etc. However, our statistical tests reveal that the energy-focused optimizations unexpectedly negatively impacted memory usage, with no clear benefits regarding execution time or energy consumption. Our qualitative analysis of energy-time trade-offs revealed that some themes, such as vectorization preallocation, were among the common themes shaping these trade-offs. With LLMs becoming ubiquitous in modern software development, our study serves as a call to action: prioritizing the evaluation of common coding practices to identify the green ones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。